Tags: , view all tags

WMS test report

WMS test report

Summary

Component: WMS 3.3.4

Savannah task: 22847

ETICS Subsystem Configuration Name: emi-wms_R_3_3_6_3 emi-jdl_R_3_2_6_1

VCS Tag: emi-ice_R_3_3_4_1, emi-wms-manager_R_3_3_1_4, emi-jobsubmission_R_3_3_2_1, emi-wmproxy_R_3_3_4_2, emi-wms-brokerinfo_R_3_3_2_1, emi-wms-configuration_R_3_3_2_1, emi-wms-ism_R_3_3_2_1, emi-wms-matchmaking_R_3_3_2_1, emi-wms-purger_R_3_3_2_1, emi-yaim-wms_R_4_1_4_3, emi-jdl-api-cpp_R_3_2_6_1, emi-wms-nagios_R_1_0_0_0, emi-wms-mp_R_1_0_2_0

EMI Major Release: EMI 1 (Kebnekaise)

Platform: sl5 epel

Certification report: See this file

Outcome: CERTIFIED

Deployment Tests

Clean installation

Starting from a SL5 x86_64 kernel machine:

[root@cream-45 ~]# cat /etc/redhat-release
Scientific Linux SL release 5.7 (Boron)
[root@cream-45 ~]# uname -a
Linux cream-45.pd.infn.it 2.6.18-238.5.1.el5xen #1 SMP Tue Mar 1 19:22:01 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

cd /etc/yum.repos.d/
wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo
yum clean all
yum install ca-policy-egi-core
yum install emi-wms
- See log file

/opt/glite/yaim/bin/yaim -c -s /usr/local/nfs/jra1_test/cream-45/site-info_wms.def -n WMS
- See log file

Update

Starting from a production version:

[root@cream-45 yum.repos.d]# etc/emi-version
1.6.0-1

wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo
yum clean all
service gLite stop
yum update
/opt/glite/yaim/bin/yaim -c -s site-info.def -n WMS
- See log file

Check final version:

[root@cream-45 ~]# cat /etc/emi-version 
1.6.0-1

Unit Tests

N/A: Tests will be available with EMI-2.

Basic Functionality tests

Submission

Normal

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 09:42:34
===> Ended  : 09:48:32
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117094234.log
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 1 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 09:59:32
===> Ended  : 10:06:54
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117095932.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:59:48
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:48
uri       : voms2.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 11:56:43
===> Ended  : 12:03:02
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117115643.log

See gzipped log file

Jdl with DataRequirements

[ale@cream-12 WMS-service]$ cat JDLs/data-req.jdl
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"JDLs/prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"};

requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/data-req.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:06:28
===> Ended  : 12:12:56
===>
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117120628.log

Test log file is here

Jdl with ISBBaseURI and OSBDestURI

Test on CREAM ce:

[ale@cream-12 DataReq]$ cat URI.jdl 
###########################################
#      JDL with ISBURI & OSBDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxDestURI = {"gsiftp://cream-44.pd.infn.it:2811/tmp/myjobOutput", "std.err"};


requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);

DataAccessProtocol = "gsiftp";
RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ cat exe.sh 
#!/bin/sh

NAME=`echo $GLITE_WMS_JOBID | sed -e 's/.*\///'`

echo $NAME 

ls -l

[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf URI.jdl

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-32.pd.infn.it:8443/cream-pbs-cert
Submitted:          Tue Nov 22 12:02:48 2011 CET
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/c0pisHDY4fELX0ycBTdNsA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_c0pisHDY4fELX0ycBTdNsA

================================================================================


[ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_c0pisHDY4fELX0ycBTdNsA
total 0
-rw-rw-r-- 1 ale ale 0 Nov 22 12:17 std.err

Log on cream-44:

[root@cream-44 ~]# cat /tmp/myjobOutput 
c0pisHDY4fELX0ycBTdNsA
total 8
-rwxr-xr-x  1 dteam002 dteam 81 Nov 22 12:03 exe.sh
-rw-r--r--  1 dteam002 dteam  0 Nov 22 12:03 std.err
-rw-r--r--  1 dteam002 dteam 23 Nov 22 12:03 std.out
-rw-r--r--  1 dteam002 dteam  0 Nov 22 12:03 test1
-rw-r--r--  1 dteam002 dteam  0 Nov 22 12:03 test2

===>   test PASSED

Same process with an LCG CE:

[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf URI.jdl

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2
Submitted:          Tue Nov 22 12:52:58 2011 CET
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/AqhMc7zmXX01vPti6o-H_A
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_AqhMc7zmXX01vPti6o-H_A

================================================================================


[ale@cream-12 DataReq]$ ls -l /tmp/jobOutput/ale_AqhMc7zmXX01vPti6o-H_A
total 0
-rw-rw-r-- 1 ale ale 0 Nov 22 12:57 std.err


Log on cream-44:


[root@cream-44 ~]# cat /tmp/myjobOutput 
AqhMc7zmXX01vPti6o-H_A
total 28
-rwxr-xr-x  1 dteam003 dteam  81 Nov 22 12:53 exe.sh
-rw-r--r--  1 dteam003 dteam 118 Nov 22 12:53 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fAqhMc7zmXX01vPti6o-H_5fA.output
-rw-r--r--  1 dteam003 dteam   0 Nov 22 12:53 std.err
-rw-r--r--  1 dteam003 dteam  23 Nov 22 12:53 std.out
-rw-r--r--  1 dteam003 dteam   0 Nov 22 12:53 test1
-rw-r--r--  1 dteam003 dteam   0 Nov 22 12:53 test2
-rw-------  1 dteam003 dteam 103 Nov 22 12:53 tmp.flIel23998
-rw-------  1 dteam003 dteam   2 Nov 22 12:53 tmp.fUCFN23997
-rw-------  1 dteam003 dteam   2 Nov 22 12:53 tmp.IZYca24012
-rw-------  1 dteam003 dteam 103 Nov 22 12:53 tmp.OLIhC24013
-rw-------  1 dteam003 dteam   0 Nov 22 12:53 tmp.RJYqf24011

===>     === test PASSED ===

Jdl with ISBBaseURI and OSBBaseDestURI

Submit to CREAM CE


[ale@cream-12 DataReq]$ cat BaseURI.jdl 
###########################################
#  JDL with ISBURI & OSBBaseDestURI       #
###########################################

Executable = "exe.sh";
StdOutput = "std.out";
StdError = "std.err";

InputSandboxBaseURI = "gsiftp://devel11.cnaf.infn.it:2811/tmp";

FuzzyRank = true;
InputSandbox = {"exe.sh", "test1", "test2"};
OutputSandbox = {"std.out", "std.err"};
OutputSandboxBaseDestURI="gsiftp://cream-44.pd.infn.it:2811/tmp";

requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf BaseURI.jdl 

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-29.pd.infn.it:8443/cream-lsf-creamcert2
Submitted:          Tue Nov 22 13:05:26 2011 CET
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA

================================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/D5jZSqbE7r94hHpKwUI8SA
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        cream-29.pd.infn.it:8443/cream-lsf-creamcert2
Submitted:          Tue Nov 22 13:05:26 2011 CET
==========================================================================


Log on cream-44:

[root@cream-44 ~]# ls -l /tmp/std.*
-rw-r--r-- 1 dteam009 dteam   0 Nov 22 13:05 /tmp/std.err
-rw-r--r-- 1 dteam009 dteam 286 Nov 22 13:05 /tmp/std.out
[root@cream-44 ~]# cat /tmp/std.out 
D5jZSqbE7r94hHpKwUI8SA
total 8
-rwxr-xr-x 1 dteam017 dteam 81 Nov 22 13:05 exe.sh
-rw-r--r-- 1 dteam017 dteam  0 Nov 22 13:05 std.err
-rw-r--r-- 1 dteam017 dteam 23 Nov 22 13:05 std.out
-rw-r--r-- 1 dteam017 dteam  0 Nov 22 13:05 test1
-rw-r--r-- 1 dteam017 dteam  0 Nov 22 13:05 test2

===>     === test PASSED ===

Submit to an LCG CE:


[ale@cream-12 DataReq]$ glite-wms-job-submit -a -c /home/ale/UI/etc/wmp_devel19.conf BaseURI.jdl 

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2
Submitted:          Tue Nov 22 13:11:26 2011 CET
==========================================================================

[ale@cream-12 DataReq]$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

No output files to be retrieved for the job:
https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA

================================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/kCtn3ymc2rgeYYn_GFD-XA
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2
Submitted:          Tue Nov 22 13:11:26 2011 CET
==========================================================================

Log on cream-44:
[root@cream-44 ~]# ls -l /tmp/std.*
-rw-r--r-- 1 dteam009 dteam   0 Nov 22 13:12 /tmp/std.err
-rw-r--r-- 1 dteam009 dteam 724 Nov 22 13:12 /tmp/std.out
[root@cream-44 ~]# cat /tmp/std.out 
kCtn3ymc2rgeYYn_GFD-XA
total 28
-rwxr-xr-x  1 dteam003 dteam  81 Nov 22 13:12 exe.sh
-rw-r--r--  1 dteam003 dteam 118 Nov 22 13:12 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fkCtn3ymc2rgeYYn_5fGFD-XA.output
-rw-r--r--  1 dteam003 dteam   0 Nov 22 13:12 std.err
-rw-r--r--  1 dteam003 dteam  23 Nov 22 13:12 std.out
-rw-r--r--  1 dteam003 dteam   0 Nov 22 13:12 test1
-rw-r--r--  1 dteam003 dteam   0 Nov 22 13:12 test2
-rw-------  1 dteam003 dteam 103 Nov 22 13:12 tmp.eOONQ24674
-rw-------  1 dteam003 dteam   2 Nov 22 13:12 tmp.qmVcP24673
-rw-------  1 dteam003 dteam   2 Nov 22 13:12 tmp.vzbZX24688
-rw-------  1 dteam003 dteam   0 Nov 22 13:12 tmp.WFASP24687
-rw-------  1 dteam003 dteam 103 Nov 22 13:12 tmp.WNCMx24689

===>     === test PASSED ===

Jdl with AllowZippedISB

[ale@cream-12 DataReq]$ cat AllowZip.jdl 
###########################################
#   JDL with AllowZippedISB attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = other.GlueCEStateStatus == "Testing" &&  regexp("/cream", other.GlueCEUniqueID);

AllowZippedISB=true;

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$  glite-wms-job-submit -a --debug  -c /home/ale/UI/etc/wmp_cream-44.conf AllowZip.jdl
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Vo read from proxy certificate extension
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - VirtualOrganisation value : dteam
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Loaded user configuration file:
 /home/ale/UI/etc/wmp_cream-44.conf
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Configuration file: //etc/glite-wms/dteam/glite_wmsui.conf - MyProxyServer attribute(s) no more supported outside JDL Default Attributes section "JdlDefaultAttributes"
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Loaded Vo specific configuration file:
 //etc/glite-wms/dteam/glite_wmsui.conf
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Loaded Configuration values: 
 [
  JdlDefaultAttributes = 
   [
    SignificantAttributes = { "Requirements","Rank" }; 
    RetryCount = 3; 
    AllowZippedISB = true; 
    VirtualOrganisation = "dteam"; 
    rank = 1; 
    MyProxyServer = "myproxy.cnaf.infn.it"
   ]; 
  OutputStorage = "/tmp"; 
  ListenerStorage = "/tmp"; 
  ErrorStorage = "/tmp"; 
  WmProxyEndPoints = { "https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server" }
 ]
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Getting Endpoint URL from configuration file 
-----------------------------------------

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server

-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getVersion service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - WMProxy Version: 3.3.4 
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Sending Proxy Request to https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy delegationns__getProxyReq service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - delegationns__getProxyReq -  The proxy has been successfully retrieved
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy delegationns__putProxyReq service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The proxy has been successfully delegated with the identifier: uehC70_noJCQm8Mb6-SBgQ
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getTransferProtocols service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - getTransferProtocols -  Available protocols: gsiftp, https
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - FileTransferProtocol not specified; using the default protocol: gsiftp
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The JDL file is: /home/ale/UI/DataReq/AllowZip.jdl
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Retrieving the list of the local ISB files from the user JDL 
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Total size of the ISB file(s) to be transferred to: 3000105
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Max single job size of the ISB file(s) to be transferred to: 2000069
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Checking the User-FreeQuota on the server 
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getFreeQuota service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - getFreeQuota -  User freequota not set on the server
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Getting the max ISB size from the server https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy getMaxInputSandboxSize service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - getMaxInputSandboxSize -  Max ISB size information successfully retrieved
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The max job size (2000069 bytes) doesn't exceed the max size limit of 100000000 bytes: File transfer is allowed
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - File archiving and file compression allowed by user in the JDL
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Registering JDL [ StdOutput = "std.out"; ShallowRetryCount = 2; SignificantAttributes = { "Requirements","Rank" }; RetryCount = 1; Executable = "/bin/ls"; Type = "job"; Arguments = "-la"; AllowZippedISB = true; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ZippedISB = { "ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz" }; OutputSandbox = { "std.out","std.err" }; InputSandbox = { "file:///home/ale/UI/DataReq/fileA","file:///home/ale/UI/DataReq/fileB","file:///home/ale/UI/DataReq/fileC","file:///home/ale/UI/DataReq/fileD","file:///home/ale/UI/DataReq/fileE" }; StdError = "std.err"; rank = 1; MyProxyServer = "myproxy.cnaf.infn.it"; requirements = other.GlueCEStateStatus == "Testing" && regexp("/cream",other.GlueCEUniqueID) ]
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Registering the job to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Calling the WMProxy jobRegister service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - The JobId is:  https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Archiving the ISB files: /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - This archive file has been successfully created: /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:20 -I- PID: 17785 (Debug) - Compressing the file (.gz): /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:21 -I- PID: 17785 (Debug) - ISB ZIPPED file successfully created: /tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:21 -I- PID: 17785 (Debug) - Getting the SandboxDestinationURI from the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:21 -I- PID: 17785 (Debug) - Calling the WMProxy getSandboxDestURI service with gsiftp protocol 
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - getSandboxDestURI -  Destination URIs sucessfully retrieved
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - DestinationURI: gsiftp://cream-44.pd.infn.it:2811/var/SandboxDir/bs/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fbsuo8b3PaOq3xF0TEzaqrw/input
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - FileTransfer (gsiftp): using globus-url-copy to transfer the local InputSandBox file(s) to the submission endpoint
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:22 -I- PID: 17785 (Debug) - File Transfer (gsiftp) 
 Command: /usr/bin/globus-url-copy
Source: file:///tmp/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz
Destination: gsiftp://cream-44.pd.infn.it:2811/var/SandboxDir/bs/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2fbsuo8b3PaOq3xF0TEzaqrw/input/ISBfiles_c4KJb6dqpljoq0YCiadrEg_0.tar.gz
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - File Transfer (gsiftp) Transfer successfully done
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - Starting the job:  https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - Calling the WMProxy jobStart service
-----------------------------------------
-----------------------------------------
22 Nov 2011, 13:34:23 -I- PID: 17785 (Debug) - jobStart -  The job has been successfully started
-----------------------------------------

====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw

==========================================================================

      *** Log file created ***
Possible Errors and Debug messages have been printed in the following file:
/tmp/glite-wms-job-submit_500_17785_20111122_13-34-20.log


Check if ZippedISB Attribute is added:

[ale@cream-12 DataReq]$ glite-wms-job-info --jdl https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw | grep Allow
  AllowZippedISB = true; 

[ale@cream-12 DataReq]$  glite-wms-job-status https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-29.pd.infn.it:8443/cream-lsf-creamcert2
Submitted:          Tue Nov 22 13:34:20 2011 CET
==========================================================================


Retrieve oputput files:
[ale@cream-12 DataReq]$ glite-wms-job-output https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-44.pd.infn.it:9000/bsuo8b3PaOq3xF0TEzaqrw
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_bsuo8b3PaOq3xF0TEzaqrw

================================================================================


Check std output to see if all the files are transfered to the CE:

[ale@cream-12 DataReq]$ cat /tmp/jobOutput/ale_bsuo8b3PaOq3xF0TEzaqrw/std.out 
total 2964
drwxr-xr-x 2 dteam017 dteam    4096 Nov 22 13:34 .
drwxr-xr-x 3 dteam017 dteam    4096 Nov 22 13:34 ..
-rw-r--r-- 1 dteam017 dteam    2282 Nov 22 13:34 .BrokerInfo
-rw-r--r-- 1 dteam017 dteam      17 Nov 22 13:34 fileA
-rw-r--r-- 1 dteam017 dteam      17 Nov 22 13:34 fileB
-rw-r--r-- 1 dteam017 dteam       0 Nov 22 13:34 fileC
-rw-r--r-- 1 dteam017 dteam 1000002 Nov 22 13:34 fileD
-rw-r--r-- 1 dteam017 dteam 2000069 Nov 22 13:34 fileE
-rw-r--r-- 1 dteam017 dteam       0 Nov 22 13:34 std.err
-rw-r--r-- 1 dteam017 dteam       0 Nov 22 13:34 std.out
-rw-r--r-- 1 dteam017 dteam       0 Nov 22 13:34 .tmp_file

===>     === test PASSED ===

Jdl with ExpiryTime

[ale@cream-12 DataReq]$ cat ExpiryTime.jdl 
###########################################
#       JDL with ExpiryTime attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = false;

ExpiryTime = 1321966000;

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ date +%s
1321965735
[ale@cream-12 DataReq]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_cream-44.conf ExpiryTime.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag

==========================================================================


[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag
Current Status:     Waiting 
Status Reason:      BrokerHelper: no compatible resources
Submitted:          Tue Nov 22 13:42:20 2011 CET
==========================================================================

[ale@cream-12 DataReq]$ date +%s
1321966381
[ale@cream-12 DataReq]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/KlcFaVR5qp7DMIRLOsz_ag
Current Status:     Aborted 
Status Reason:      request expired
Submitted:          Tue Nov 22 13:42:20 2011 CET
==========================================================================

===>     === test PASSED ===

Jdl with ShortDeadlineJob

[ale@cream-12 DataReq]$ cat JDLs/SDLJ.jdl 
###########################################
# JDL with ShortDeadlineJob attribute     #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";

InputSandbox = {"fileA", "fileB", "fileC", "fileD", "fileE"};
OutputSandbox = {"std.out", "std.err"};

requirements = true;

ShortDeadlineJob = true;

RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 WMS-service]$ glite-wms-job-list-match -a   -c /home/ale/UI/etc/wmp_devel19.conf  JDLs/SDLJ.jdl

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce01.ariagni.hellasgrid.gr:2119/jobmanager-pbs-dteam.sdj
 - ce01.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
 - ce02.marie.hellasgrid.gr:2119/jobmanager-pbs-sdj
 - cream-ce01.ariagni.hellasgrid.gr:8443/cream-pbs-dteam.sdj
 - cream-ce01.marie.hellasgrid.gr:8443/cream-pbs-sdj
 - grid4.mif.vu.lt:2119/jobmanager-lcgpbs-sdj
 - grive02.ibcp.fr:8443/cream-pbs-sdj
 - mario.univ-lille1.fr:8443/cream-pbs-sdj

==========================================================================

[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 1 -i -j JDLs/SDLJ.jdl
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 1: Submit a normal job (3 cases LCG-CE, CREAM, general) 


 Test 1A: Submit to an LCG-CE 


 Test 1B: Submit to a CREAM CE 


 Test 1C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: devel19.cnaf.infn.it
===> Started: 15:47:17
===> Ended  : 15:55:47
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111122154717.log

Test log file is here

Collection

Bulk submission


[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:13:08
===> Ended  : 12:19:10
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117121308.log
You have new mail in /var/spool/mail/ale
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 2 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 15:38:31
===> Ended  : 15:44:31
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117153831.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 6:49:04
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 7:48:04
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 2 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 2: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) 


 Test 2A: Submit to an LCG-CE 


 Test 2B: Submit to a CREAM CE 


 Test 2C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 15:45:22
===> Ended  : 15:53:23
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117154522.log

See gzipped log file

All in a jdl

[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 16:51:46
===> Ended  : 16:59:22
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117165146.log
[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -t 7 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:00:03
===> Ended  : 17:06:41
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117170003.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 5:46:00
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 6:45:00
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 7 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 7: Submit a bulk of jobs (3 cases LCG-CE, CREAM, general) using a single jdl with al the jdls of nodes 


 Test 7A: Submit to an LCG-CE 


 Test 7B: Submit to a CREAM CE 


 Test 7C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:10:15
===> Ended  : 17:16:25
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117171015.log

See gzipped log file

Parametric job

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:21:01
===> Ended  : 17:29:29
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117172101.log
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu  -d 3 -l -t 3 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:31:41
===> Ended  : 17:40:42
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117173141.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 5:08:06
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 6:07:06
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 3 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 3: Submit a parametric job (3 cases LCG-CE, CREAM, general) 


 Test 3A: Submit to an LCG-CE 


 Test 3B: Submit to a CREAM CE 


 Test 3C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:48:08
===> Ended  : 17:54:45
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117174808.log

See gzipped log file

DAG job

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:58:23
===> Ended  : 18:03:39
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117175823.log
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu  -d 3 -l -t 4 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 18:04:03
===> Ended  : 18:09:16
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117180403.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 4:45:13
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 5:44:13
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 4 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 4: Submit a DAG job 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 18:11:01
===> Ended  : 18:16:12
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117181101.log

See gzipped log file

Parallel job

[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) 


 Test 5A: Submit to an LCG-CE 


 Test 5B: Submit to a CREAM CE 


 Test 5C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 13:25:11
===> Ended  : 13:36:07
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118132511.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:00:27
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 10:59:27
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$  python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 5 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 5: Submit a MPI job (3 cases LCG-CE, CREAM, general) 


 Test 5A: Submit to an LCG-CE 


 Test 5B: Submit to a CREAM CE 


 Test 5C: Submit without restrictions 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 15:18:11
===> Ended  : 15:24:04
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118151811.log

See gzipped log file

Testing forwarding parameters for parallel jobs

In order to test the passing of mpi parameters (wholenodes,smpgranularity,cpunumber,hostnumber) to the ce, an mpi jdl is submitted with varying values for the parameters, specifying a cream CE as destination. Then the cream jobid is retrieved via a glite-wms-job-logging-info -v 2 ("Dest jobid" field of a Transfer event).Finally the cream jdl is found via glite-ce-job-status -L 2 <cream-jobid>.

The following jdls were submitted:

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;
CpuNumber=2;
Hostnumber = 1;

Resultim cream jdl:

JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_IggVNRt3plhKwsv7zkIQRw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/gQF7dCwrxP9pfDne1RfyOA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 2; edg_previous_matches = { "ce.scope.unina.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319039372; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 1; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 2 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/gQ/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fgQF7dCwrxP9pfDne1RfyOA" ]]

jdl:

CpuNumber=1;
Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 2;

Resulting cream jdl:

JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000011:BH=0000000000:JSS=000004:LM=000012:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_A4rYVNupOqd7YmRathOHog_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/94xFQjidomDV4P_pzEey5A"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 2; InputSandboxPath = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 1; edg_previous_matches = { "gridsrv2-4.dir.garr.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319037684; ce_id = "gridsrv2-4.dir.garr.it" ] }; X509UserProxy = "/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 1 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 1 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/94/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2f94xFQjidomDV4P_5fpzEey5A" ]]

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;
Hostnumber = 1;

Submission result:

[fabiok@ui mpi]$ glite-wms-job-submit -a -c ../wms_cream-45.conf mpi.jdl 

Connecting to the service https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server


Warning - Unable to register the job to the service: https://cream-45.pd.infn.it:7443/glite_wms_wmproxy_server
SMPGranularity and HostNumber are mutually exclusive when WholeNodes allocation is not requested: wrong combination of values

Method: jobRegister


Error - Operations failed
Unable to find any endpoint where to perform service request

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = false;
SMPGranularity = 3;
CpuNumber=3;

Resulting cream jdl:

JDL            = [[ CEInfoHostName = "ce-02.roma3.infn.it"; OutputSandboxPath = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "ce-02.roma3.infn.it:8443/cream-pbs"; QueueName = "cert"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000017:BH=0000000000:JSS=000006:LM=000022:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "ce-02.roma3.infn.it:8443/cream-pbs-cert"; ZippedISB = { "ISBfiles_km6imNWXB17PDZruVO82Mg_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/n2xH3q9eUcG1o0x9ZgFNEA"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 3; InputSandboxPath = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/input"; CeApplicationDir = "/storage/local/exp_soft/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = false; CpuNumber = 3; edg_previous_matches = { "grisuce.scope.unina.it","ce.scope.unina.it" }; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; edg_previous_matches_ex = { [ timestamp = 1319039871; ce_id = "grisuce.scope.unina.it" ],[ timestamp = 1319039874; ce_id = "ce.scope.unina.it" ] }; X509UserProxy = "/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 3 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/n2/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fn2xH3q9eUcG1o0x9ZgFNEA" ]]

jdl:

Executable = "openmpi-wrapper.sh";
Arguments="hello";
StdOutput  = "hello.out";
StdError   = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
InputSandbox = {"/home/fabiok/mpi/openmpi-wrapper.sh","/home/fabiok/mpi/hello.c","/home/fabiok/mpi/Makefile"};
Requirements = Member ("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*", other.GlueCEUniqueID);
WholeNodes = true;
SMPGranularity = 8;
CpuNumber=8;
Hostnumber = 2;

Resulting cream jdl:

JDL            = [[ CEInfoHostName = "prabi-ce5-gpu.ibcp.fr"; OutputSandboxPath = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output"; StdOutput = "hello.out"; BatchSystem = "pbs"; GlobusResourceContactString = "prabi-ce5-gpu.ibcp.fr:8443/cream-pbs"; QueueName = "dteam"; ShallowRetryCount = 10; SignificantAttributes = { "Requirements","Rank","FuzzyRank" }; RetryCount = 3; Executable = "openmpi-wrapper.sh"; Type = "job"; LB_sequence_code = "UI=000000:NS=0000000004:WM=000005:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000"; Arguments = "hello"; AllowZippedISB = true; maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation = "dteam"; JobType = "normal"; DefaultRank =  -other.GlueCEStateEstimatedResponseTime; ce_id = "prabi-ce5-gpu.ibcp.fr:8443/cream-pbs-dteam"; ZippedISB = { "ISBfiles_4xXBPYiJ8Z6ZeIyiIUREnw_0.tar.gz" }; OutputSandboxDestURI = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output/hello.out","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/output/hello.err" }; OutputSandbox = { "hello.out","hello.err" }; CeRequirements = "( true && ( true && ( true && ( true && ( true && ( true && Member(\"OPENMPI\",other.GlueHostApplicationSoftwareRunTimeEnvironment) ) ) ) ) ) ) && ( true && ( ( ShortDeadlineJob is true ) ? false : true ) && ( EnableWmsFeedback is true ? true : true ) )"; InputSandbox = { "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/openmpi-wrapper.sh","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/hello.c","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/Makefile","gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input/.BrokerInfo" }; edg_jobid = "https://cream-45.pd.infn.it:9000/HsKoKN4FdQ3oGzyhzrxLuw"; VOMS_FQAN = "/dteam/Role=NULL/Capability=NULL"; CertificateSubject = "/C=IT/O=INFN/OU=Personal Certificate/L=CNAF/CN=Fabio Capannini"; StdError = "hello.err"; SMPGranularity = 8; InputSandboxPath = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/input"; CeApplicationDir = "/swareas/dteam"; ReallyRunningToken = "gsiftp://cream-45.pd.infn.it/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/token.txt_0"; rank =  -other.GlueCEStateEstimatedResponseTime; WholeNodes = true; CpuNumber = 8; WMSHostname = "cream-45.pd.infn.it"; lrms_type = "pbs"; MyProxyServer = "myproxy.cnaf.infn.it"; X509UserProxy = "/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw/user.proxy"; InputSandboxDestFileName = { "openmpi-wrapper.sh","hello.c","Makefile" }; Hostnumber = 2; requirements = ( ( ( ( ( ( Member("OPENMPI",other.GlueHostApplicationSoftwareRunTimeEnvironment) && RegExp(".*cream.*",other.GlueCEUniqueID) ) && ( other.GlueCEStateStatus == "Production" ) ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && other.GlueCEInfoTotalCPUs >= 8 ) && other.GlueHostArchitectureSMPSize >= SMPGranularity ) && ( ( ( ShortDeadlineJob is true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :  !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && ( EnableWmsFeedback is true ? RegExp("cream",other.GlueCEImplementationName,"i") : true ) ); WMPInputSandboxBaseURI = "gsiftp://cream-45.pd.infn.it:2811/var/SandboxDir/Hs/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fHsKoKN4FdQ3oGzyhzrxLuw" ]]

===> test PASSED

Perusal job

[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 18:18:54
===> Ended  : 18:46:20
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111117181854.log
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V testers.eu-emi.eu  -d 3 -l -t 6 -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 09:46:54
===> Ended  : 10:14:18
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118094654.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:59:57
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:57
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-job-cycle.py -c wms-command.conf -V dteam -d 3 -l -t 6 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test a complete job cycle: from submission to get output 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Set 6: Submit a perusal job 


 BEWARE default min perusal interval is 1000 secs, so this phase could take many minutes 

===> 
===> Test: WMS-job-cycle.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:45:02
===> Ended  : 13:11:24
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118124502.log

See gzipped log file

Delegation

[ale@cream-12 WMS-service]$  python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 Test 3: Try to delegate with a short proxy and check the validity 


 Test 4: Works with expired proxy 

===> 
===> Test: WMS-delegation-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:05:09
===> Ended  : 17:06:24
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118170509.log
[ale@cream-12 WMS-service]$  python WMS-delegation-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 Test 3: Try to delegate with a short proxy and check the validity 


 Test 4: Works with expired proxy 

===> 
===> Test: WMS-delegation-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:06:50
===> Ended  : 17:08:00
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118170650.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:59:55
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:58:55
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$  python WMS-delegation-test.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: Test delegation operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Check --autm-delegation option 


 Test 2: Check --delegationid option 


 There are other two tests which require the user proxy password. Use -i option to enable them 

===> 
===> Test: WMS-delegation-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:19:56
===> Ended  : 17:20:00
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118171956.log

See gzipped log file

Job List-Match

Without data

[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 


 Test 6: try a matching with EnableWMSFeedback attribute true 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:49:39
===> Ended  : 17:49:49
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118174939.log
[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 


 Test 6: try a matching with EnableWMSFeedback attribute true 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:49:51
===> Ended  : 17:50:02
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118174951.log
[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 10:29:30
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 11:28:30
uri       : voms2.hellasgrid.gr:15004
[ale@cream-12 WMS-service]$ python WMS-listmatch-test.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job ListMatch Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: check if glite-wms-job-list-match command exists 


 Test 2: test a simple matching 


 Test 3: check --delegationid option 


 Test 4: exclude a CE from the match 


 Test 5: try a failure matching (Requirements == false) 


 Test 6: try a matching with EnableWMSFeedback attribute true 

===> 
===> Test: WMS-listmatch-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:50:19
===> Ended  : 17:50:25
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118175019.log

See gzipped log file

With data

Two files have been registered on the SE srm-public.cern.ch:

find an lfc for the dteam vo:
lcg-infosites --vo dteam lfc

set the lfc host:
export LFC_HOST=prod-lfc-shared-central.cern.ch

create the directory:
lfc-mkdir /grid/dteam/fcapanni

create the files:
lcg-cr --vo dteam -d srm://srm-public.cern.ch:8443/castor/cern.ch/grid/dteam/fcapanni/e-2M.txt -l lfn:/grid/dteam/fcapanni/e-2M.txt "file://$PWD/e-2M.txt"
lcg-cr --vo dteam -d srm://srm-public.cern.ch:8443/castor/cern.ch/grid/dteam/fcapanni/PI_1M.txt -l lfn:/grid/dteam/fcapanni/PI_1M.txt "file://$PWD/PI_1M.txt"

Then the following jdl was submitted:

###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "calc-pi.sh";
# Arguments is the number of digits, must be < 1000000
Arguments = "1000";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;

InputSandbox = {"calc-pi.sh", "fileA", "fileB","prologue.sh"};
OutputSandbox = {"std.out", "std.err","out-PI.txt","out-e.txt"};

#Requirements = other.GlueCEInfoHostName != "spacin-ce1.dma.unina.it";
Requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog = "http://prod-lfc-shared-central.cern.ch:8085";
InputData = {"lfn:/grid/dteam/fcapanni/PI_1M.txt","lfn:/grid/dteam/fcapanni/e-2M.txt"};
]
};

DataAccessProtocol = "gsiftp";
VirtualOrganisation = "dteam";

RetryCount = 1;
ShallowRetryCount = 2;

The job was assigned to the CE ce133.cern.ch.

Verification that the SE srm-public.cern.ch is close to the CE ce133.cern.ch:

[fabiok@ui DataReq]$ lcg-infosites --vo dteam closeSE|grep -A 5 ce133
Name of the CE: ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
    srm-alice.cern.ch
    srm-atlas.cern.ch
    srm-cms.cern.ch
    srm-lhcb.cern.ch
    srm-public.cern.ch

===> test PASSED

Gangmatching

Tried a list-match considering the following JDL:

[
executable="ssh1.sh";
myproxyserver="";
SErequirements = anyMatch(other.storage.CloseSEs, target.GlueSAStateAvailableSpace > 20);
CErequirements = other.GlueCEStateStatus == "Production" && other.GlueCEInfoTotalCPUs >= 2;
requirements = CErequirements && SErequirements;
inputsandbox={"ssh1.sh"};
stdoutput="out3.out";
stderror="err2.err";
outputsandbox={"out3.out", "err2.err", "ssh1.sh", "a"};
]


$ glite-wms-job-list-match -a -c cream44.conf gang.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - cccreamceli05.in2p3.fr:8443/cream-sge-long
 - cccreamceli05.in2p3.fr:8443/cream-sge-medium
 - cccreamceli05.in2p3.fr:8443/cream-sge-short
 - cccreamceli06.in2p3.fr:8443/cream-sge-long
 - cccreamceli06.in2p3.fr:8443/cream-sge-medium
 - cccreamceli06.in2p3.fr:8443/cream-sge-short
 - cccreamceli10.in2p3.fr:8443/cream-sge-long
 - cccreamceli10.in2p3.fr:8443/cream-sge-medium
 - cccreamceli10.in2p3.fr:8443/cream-sge-short
 - cmsce01.na.infn.it:2119/jobmanager-lcgpbs-cert
 - grid010.ct.infn.it:2119/jobmanager-lcgpbs-cert
 - inaf-ce-01.ct.pi2s2.it:8443/cream-lsf-cert
 - virgo-ce.roma1.infn.it:2119/jobmanager-lcgpbs-cert
 - cccreamceli09.in2p3.fr:8443/cream-sge-long
 - cccreamceli09.in2p3.fr:8443/cream-sge-medium
 - cccreamceli09.in2p3.fr:8443/cream-sge-short
 - ce-01.grid.sissa.it:8443/cream-pbs-blade
 - cccreamceli02.in2p3.fr:8443/cream-bqs-short
 - cccreamceli02.in2p3.fr:8443/cream-bqs-medium
 - unict-dmi-ce-01.ct.pi2s2.it:2119/jobmanager-lcglsf-cert
 - cream-ce.pg.infn.it:8443/cream-pbs-cert

==========================================================================

Manually checked in the bdii some of matched CE and verified that they really meet the constraints.

Tried also modifying the JDL setting:

requirements = CErequirements;

Many more CEs get matched with a list-match.

===> test PASSED

Prologue and Epilogue jobs

[ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l  -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: cream-44.pd.infn.it
===> Started: 17:56:10
===> Ended  : 18:03:36
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118175610.log
[ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: cream-44.pd.infn.it
===> Started: 18:15:28
===> Ended  : 18:21:51
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111118181528.log
[ale@cream-12 WMS-service]$ python WMS-prologue-epilogue.py -c wms-command.conf -V dteam -d 3 -l  
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test prologue and epilogue attributes 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test prologue attribute 


 Test 2: Test epilogue attribute 


 Test 3: Test prologue and epilogue attributes 

===> 
===> Test: WMS-prologue-epilogue.py
===> WMS: cream-44.pd.infn.it
===> Started: 11:59:49
===> Ended  : 12:07:40
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121115949.log

See gzipped log file

Shallow and deep re-submission

[ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:17:57
===> Ended  : 12:28:09
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121121757.log
You have new mail in /var/spool/mail/ale
[ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:28:27
===> Ended  : 12:40:38
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121122827.log
[ale@cream-12 WMS-service]$ python WMS-job-resubmit.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Resubmission Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Try a shallow resubmission 


 Test 2: Try a deep resubmission 

===> 
===> Test: WMS-job-resubmit.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:40:42
===> Ended  : 12:49:48
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121124042.log

See gzipped log file

WMS Job Cancel Testing

[ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: devel19.cnaf.infn.it
===> Started: 13:21:24
===> Ended  : 13:29:22
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121132124.log
[ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: devel19.cnaf.infn.it
===> Started: 13:30:54
===> Ended  : 13:39:55
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121133054.log

[ale@cream-12 WMS-service]$ voms-proxy-info -all
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
issuer    : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
identity  : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy
type      : proxy
strength  : 2048 bits
path      : /tmp/x509up_u500
timeleft  : 7:51:40
key usage : Digital Signature, Key Encipherment, Data Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
attribute : /dteam/NGI_IT/Role=NULL/Capability=NULL
timeleft  : 8:50:40
uri       : voms2.hellasgrid.gr:15004

[ale@cream-12 WMS-service]$ python WMS-job-cancel.py -c wms-command.conf -V dteam -d 3 -l  
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Job Cancel Testing 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Try to cancel a normal job 


 Test 2: Try to cancel a DAG job 


 Test 3: Try to cancel one node of a DAG job 


 Test 4: Try to cancel a collection job 


 Test 5: Try to cancel one node of a collection job 


 Test 6: Try to cancel a parametric job 


 Test 7: Try to cancel one node of a parametric job 


 Test 8: Try to cancel a MPI job 

===> 
===> Test: WMS-job-cancel.py
===> WMS: devel19.cnaf.infn.it
===> Started: 14:41:56
===> Ended  : 14:50:00
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111121144156.log

See gzipped log file

Proxy renewal

[ale@cream-12 WMS-service]$ python WMS-proxy-renewal.py -c wms-command2.conf -V dteam -d 3 -l  -i 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test proxy renewal operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test proxy renewal with LCG CE 


 Test 2: Test proxy renewal with CREAM CE 


 Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) 


 Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) 

===> ERROR: Timeout reached while waiting the job https://devel11.cnaf.infn.it:9000/ezPwvHZibxuCyErO_FIauQ to finish
===> 
===> Test: WMS-proxy-renewal.py
===> WMS: devel11.cnaf.infn.it
===> Started: 10:36:38
===> Ended  : 11:47:05
===> 
===>     >>> TEST FAILED <<<
===> 
===>  >>> failure reason: 1 test(s) fail(s): ['Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER)'] <<< 
===> 
===> Test log file is WMSService-TS_20111122103638.log
===> Error messages have been written in /home/ale/wms_tests/WMS-service/WMSService-Test_20111122103638/errors.log
===> 
===>  Test directory /home/ale/wms_tests/WMS-service/WMSService-Test_20111122103638 has not been cleaned for debug purpose




[ale@cream-12 WMS-service]$  python WMS-proxy-renewal.py -c wms-command.conf -V testers.eu-emi.eu -d 3 -l  -i
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS test proxy renewal operation 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

Enter the user proxy password:

 Test 1: Test proxy renewal with LCG CE 


 Test 2: Test proxy renewal with CREAM CE 


 Test 3: Test proxy renewal with LCG CE (without setting MYPROXYSERVER) 


 Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER) 

===> ERROR: Timeout reached while waiting the job https://devel19.cnaf.infn.it:9000/ekn6N9tFatImb64X7Rwy6Q to finish
===> 
===> Test: WMS-proxy-renewal.py
===> WMS: devel19.cnaf.infn.it
===> Started: 10:18:16
===> Ended  : 11:30:09
===> 
===>     >>> TEST FAILED <<<
===> 
===>  >>> failure reason: 1 test(s) fail(s): ['Test 4: Test proxy renewal with CREAM CE (without setting MYPROXYSERVER)'] <<< 
===> 
===> Test log file is WMSService-TS_20111122101816.log
===> Error messages have been written in /home/ale/wms_tests/WMS-service/WMSService-Test_20111122101816/errors.log
===> 
===>  Test directory /home/ale/wms_tests/WMS-service/WMSService-Test_20111122101816 has not been cleaned for debug purpose

NB Last Test fail for bug #84839

See gzipped log file

Configuration file

Verified that in the configuration file /etc/glite-wms/glite_wms.conf there are these hard-coded values:

For the common section:

DGUser = "\${GLITE_WMS_USER}"
HostProxyFile = "\${WMS_LOCATION_VAR}/glite/wms.proxy"
LBProxy = true

For the JobController section:

CondorSubmit = "${CONDORG_INSTALL_PATH}/bin/condor_submit"
CondorRemove = "${CONDORG_INSTALL_PATH}/bin/condor_rm"
CondorQuery = "${CONDORG_INSTALL_PATH}/bin/condor_q"
CondorRelease = "${CONDORG_INSTALL_PATH}/bin/condor_release"
CondorDagman = "${CONDORG_INSTALL_PATH}/bin/condor_dagman"
DagmanMaxPre = 10
SubmitFileDir = "${WMS_LOCATION_VAR}/jobcontrol/submit"
OutputFileDir = "${WMS_LOCATION_VAR}/jobcontrol/condorio"
InputType = "jobdir"
Input = "${WMS_LOCATION_VAR}/jobcontrol/jobdir/"
LockFile = "${WMS_LOCATION_VAR}/jobcontrol/lock"
LogFile = "\${WMS_LOCATION_LOG}/jobcontoller_events.log"
LogLevel = 5
MaximumTimeAllowedForCondorMatch = 1800
ContainerRefreshThreshold = 1000

For the NetworkServer section:

II_Port  = 2170
Gris_Port = 2170
II_Timeout = 100
Gris_Timeout = 20
II_DN = "mds-vo-name=local, o=grid"
Gris_DN = "mds-vo-name=local, o=grid"
BacklogSize = 64
ListeningPort = 7772
MasterThreads = 8
DispatcherThreads = 10
SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir"
LogFile = "${WMS_LOCATION_LOG}/networkserver_events.log"
LogLevel = 5
EnableQuotaManagement = false
MaxInputSandboxSize = 10000000
EnableDynamicQuotaAdjustment = false
QuotaAdjustmentAmount = 10000
QuotaInsensibleDiskPortion = 2.0
DLI_SI_CatalogTimeout = 60
ConnectionTimeout = 300

For the LogMonitor section:

JobsPerCondorLog = 1000
LockFile = "${WMS_LOCATION_VAR}/logmonitor/lock"
LogFile = "${WMS_LOCATION_LOG}/logmonitor_events.log"
LogLevel = 5
ExternalLogFile = "\${WMS_LOCATION_LOG}/logmonitor_external.log"
MainLoopDuration = 5
CondorLogDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log"
CondorLogRecycleDir = "${WMS_LOCATION_VAR}/logmonitor/CondorG.log/recycle"
MonitorInternalDir = "${WMS_LOCATION_VAR}/logmonitor/internal"
IdRepositoryName = "irepository.dat"
AbortedJobsTimeout = 600
GlobusDownTimeout = 7200
RemoveJobFiles = true
ForceCancellationRetries = 2

For the Workloadmanager section:

PipeDepth = 200
WorkerThreads = 5
DispatcherType = "jobdir"
Input = "${WMS_LOCATION_VAR}/workload_manager/jobdir"
LogLevel = 5
LogFile  = "${WMS_LOCATION_LOG}/workload_manager_events.log"
MaxRetryCount = 10
CeMonitorServices = {}
CeMonitorAsynchPort = 0
IsmBlackList = {}
IsmUpdateRate = 600
IsmIiPurchasingRate = 480
JobWrapperTemplateDir = "${WMS_JOBWRAPPER_TEMPLATE}"
IsmThreads = false
IsmDump = "${WMS_LOCATION_VAR}/workload_manager/ismdump.fl"
SiServiceName = "org.glite.SEIndex"
DliServiceName = "data-location-interface"
MaxRetryCount = 10
DisablePurchasingFromGris = true
EnableBulkMM = true
CeForwardParameters = {"GlueHostMainMemoryVirtualSize","GlueHostMainMemoryRAMSize","GlueCEPolicyMaxCPUTime"}
MaxOutputSandboxSize = -1
EnableRecovery = true
QueueSize = 1000
ReplanGracePeriod = 3600
MaxReplansCount = 5
WmsRequirements  = ((ShortDeadlineJob =?= TRUE) ? RegExp(".*sdj$", other.GlueCEUniqueID) : !RegExp(".*sdj$", other.GlueCEUniqueID)) && (other.GlueCEPolicyMaxTotalJobs == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs) && (EnableWmsFeedback =?= TRUE ? RegExp("cream", other.GlueCEImplementationName, "i") : true)

For the WorkloadManagerProxy:

SandboxStagingPath = "${WMS_LOCATION_VAR}/SandboxDir"
LogFile = "${WMS_LOCATION_LOG}/wmproxy.log"
LogLevel = 5
MaxInputSandboxSize = 100000000
ListMatchRootPath = "/tmp"
GridFTPPort = 2811
LBLocalLogger = "localhost:9002"
MinPerusalTimeInterval = 1000
AsyncJobStart = true
EnableServiceDiscovery = false
LBServiceDiscoveryType = "org.glite.lb.server"
ServiceDiscoveryInfoValidity = 3600
WeightsCacheValidity = 86400
MaxServedRequests = 50
OperationsLoadScripts = [
jobRegister = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobRegister --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 1500 --ftpconn 300"
jobSubmit = "${WMS_LOCATION_SBIN}/glite_wms_wmproxy_load_monitor --oper jobSubmit --load1 22 --load5 20 --load15 18 --memusage 99 --diskusage 95 --fdnum 1000 --jdnum 1500  --ftpconn 300"
RuntimeMalloc = "/usr/lib64/libtcmalloc_minimal.so"
]

For the ICE section:

start_listener  =  false
start_lease_updater  =  false
logfile  =  "${WMS_LOCATION_LOG}/ice.log"
log_on_file = true
creamdelegation_url_prefix  =  "https://"
listener_enable_authz  =  true
poller_status_threshold_time  =  30*60
ice_topic  =  "CREAM_JOBS"
subscription_update_threshold_time  =  3600
lease_delta_time  =  0
notification_frequency  =  3*60
start_proxy_renewer  =  true
max_logfile_size  =  100*1024*1024
ice_host_cert  =  "${GLITE_HOST_CERT}"
Input  =  "${WMS_LOCATION_VAR}/ice/jobdir"
job_cancellation_threshold_time  =  300
poller_delay  =  2*60
persist_dir  =  "${WMS_LOCATION_VAR}/ice/persist_dir"
lease_update_frequency  =  20*60
log_on_console = false
cream_url_postfix  =  "/ce-cream/services/CREAM2"
subscription_duration  =  86400
bulk_query_size  =  100
purge_jobs  =  false
InputType  =  "jobdir"
listener_port  =  7010
listener_enable_authn  =  true
ice_host_key  =  "${GLITE_HOST_KEY}"
start_poller  =  true
creamdelegation_url_postfix  =  "/ce-cream/services/gridsite-delegation"
cream_url_prefix  =  "https://"
max_ice_threads  =  10
cemon_url_prefix  =  "https://"
start_subscription_updater  =  true
proxy_renewal_frequency  =  600
ice_log_level  =  700
soap_timeout  =  60
max_logfile_rotations  =  10
cemon_url_postfix  =  "/ce-monitor/services/CEMonitor"
max_ice_mem = 2096000
ice_empty_threshold = 600

Verified that:

The attribute II_Contact of NetworkServer section matches the value of the yaim variable BDII_HOST
The attribute WMExpiryPeriod of WorkloadManager section matches the value of yaim variable WMS_EXPIRY_PERIOD
The attribute MatchRetryPeriod of WorkloadManager section matches the value of yaim variable WMS_MATCH_RETRY_PERIOD
The attribute IsmIiLDAPCEFilterExt of WorkloadManager section is (|(GlueCEAccessControlBaseRule=VO:vo1)(GlueCEAccessControlBaseRule=VOMS:/vo1/*)(GlueCEAccessControlBaseRule=VO=vo2...
The attribute LBServer of the WorkloadManagerProxy section matches the value of yaim variable LB_HOST

===> test PASSED

WMS feedback

[ale@cream-12 WMS-service]$ python WMS-feedback-test.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Feedback 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test WMS feedback feature 

===> 
===> Test: WMS-feedback-test.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:22:27
===> Ended  : 12:25:53
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111123122227.log

Test log file is here

Limiter mechanism

[ale@cream-12 WMS-service]$ python WMS-limiter-mechanism.py -c wms-command.conf -V dteam -d 3 -l -t 1,4,6-10
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Limiter Mechanism 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test option --load1 threshold for load average (1 min) 


 Test 4: Test option --memusage threshold for memory usage (%) 


 Test 6: Test option --ftpconn threshold for number of FTP connections 


 Test 7: Test option --diskusage threshold for disk usage (%) 


 Test 8: Test option --fdnum threshold for used file descriptor 


 Test 9: Test option --jdnum threshold for number of unprocessed jobs (for jobdir) 


 Test 10: Test option --jdsize threshold for input jobdir size (KB) 

===> 
===> Test: WMS-limiter-mechanism.py
===> WMS: cream-44.pd.infn.it
===> Started: 11:59:45
===> Ended  : 12:00:15
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111123115945.log

Test log file is here

Purge

Normal job cycle

[ale@cream-12 WMS-service]$ python WMS-check-purger.py -c wms-command.conf -V dteam -d 3 -l 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++
===> + TestSuite of the WMS Service                       
===> + Description: WMS Purge Mechanism 
===> +++++++++++++++++++++++++++++++++++++++++++++++++++++

 Test 1: Test purge , normal job cycle ( Submit to LCG CE ) 


 Test 2: Test purge, normal job cycle ( Submit to CREAM CE ) 


 Test 3: Test proxy cache purging 

===> 
===> Test: WMS-check-purger.py
===> WMS: cream-44.pd.infn.it
===> Started: 12:57:02
===> Ended  : 13:09:03
===> 
===>     === test PASSED === 
===> 
===> Test log file is WMSService-TS_20111123125702.log

Test log file is here

SandBoxDir cron purger

The purge of the SandBox dir is done by a cron job

- Submit a job and force its purge before it finishes:

[ale@cream-12 UI]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_devel19.conf test.jdl

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w

==========================================================================

[ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        cream-24.pd.infn.it:8443/cream-lsf-testbedB_1
Submitted:          Wed Nov 23 13:07:12 2011 CET
==========================================================================

[root@devel19 ~]# su - glite
[glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/qD -s
23 Nov, 13:08:01 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w: removed SCHEDULED job

[ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/qDSMCo6BvbAurBg7uy-R_w
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Destination:        cream-24.pd.infn.it:8443/cream-lsf-testbedB_1
Submitted:          Wed Nov 23 13:07:12 2011 CET
==========================================================================

===>     === test PASSED === 

- Purge a done job:

[ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-23.pd.infn.it:8443/cream-lsf-cert
Submitted:          Sat Nov 19 15:03:32 2011 CET
Parent Job:         https://devel19.cnaf.infn.it:9000/jfna3yAaluQf4CfBZp38tQ
==========================================================================

[glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/ZZ  | grep ZZ7OoZhwB-6tvBaxTrheVw
23 Nov, 13:10:25 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw: removed DONE job


[ale@cream-12 UI]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/ZZ7OoZhwB-6tvBaxTrheVw
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Destination:        cream-23.pd.infn.it:8443/cream-lsf-cert
Submitted:          Sat Nov 19 15:03:32 2011 CET
Parent Job:         https://devel19.cnaf.infn.it:9000/jfna3yAaluQf4CfBZp38tQ
==========================================================================

- Purge only jobs older than 12 hours:

[glite@devel19 ~]$ ls -l /var/SandboxDir/9O
total 68
drwxrwx--- 5 dteam042 glite 4096 Nov 20 03:19 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O0DOmLWZKbIix2SZrBQBA
drwxrwx--- 5 dteam042 glite 4096 Nov 20 08:55 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O1zK64Gl9qEaPFw-LHPbQ
drwxrwx--- 5 dteam042 glite 4096 Nov 20 15:56 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O2NaEZENXOsQ5ARurVy9Q
drwxrwx--- 5 dteam042 glite 4096 Nov 19 21:11 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9O46dhW7QO5mihD_5f3arqUg
drwxrwx--- 5 dteam002 glite 4096 Nov 19 08:30 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OAG_5fKMvemu6G6_5fh1QtsXw
drwxrwx--- 5 dteam002 glite 4096 Nov 19 03:48 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9Of9YBbvsamRURogbFCTog
drwxrwx--- 5 dteam042 glite 4096 Nov 19 10:15 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OIA4NvNIy_5fFqGDxSgq36A
drwxrwx--- 5 dteam002 glite 4096 Nov 20 06:43 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OJKWi0W9tAqMEvk1KMpXg
drwxrwx--- 5 dteam002 glite 4096 Nov 18 21:02 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OJpHAsHmN68qRMgRR79bQ
drwxrwx--- 5 dteam002 glite 4096 Nov 19 18:42 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OkIA_5fHWdJYQ2jhWtzN2qQ
drwxrwx--- 5 dteam042 glite 4096 Nov 23 13:19 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OMsNaP2bzC3oiToyAu5hA
drwxrwx--- 5 dteam042 glite 4096 Nov 20 02:29 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OoF8o2i76X5cOEWO9_5fezw
drwxrwx--- 5 dteam002 glite 4096 Nov 19 08:18 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OPevUO6LxxqDeoyUjrgjw
drwxrwx--- 5 dteam042 glite 4096 Nov 20 00:28 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OsqPkEq2A0BjaxsS83DxQ
drwxrwx--- 5 dteam042 glite 4096 Nov 20 07:54 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OvoOJROG6hsz-dAJsAGRQ
drwxrwx--- 5 dteam042 glite 4096 Nov 20 14:26 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OWbkU0KtgP1fWYqA2SHuw
drwxrwx--- 5 dteam002 glite 4096 Nov 20 08:34 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OYsYtPw4B6qBu-N3G4cNw
[glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/9O -t 43200
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O1zK64Gl9qEaPFw-LHPbQ: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OIA4NvNIy_FqGDxSgq36A: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O2NaEZENXOsQ5ARurVy9Q: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OYsYtPw4B6qBu-N3G4cNw: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OsqPkEq2A0BjaxsS83DxQ: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OkIA_HWdJYQ2jhWtzN2qQ: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OJKWi0W9tAqMEvk1KMpXg: removed DONE job
23 Nov, 13:19:43 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OAG_KMvemu6G6_h1QtsXw: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OoF8o2i76X5cOEWO9_ezw: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OJpHAsHmN68qRMgRR79bQ: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OvoOJROG6hsz-dAJsAGRQ: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OPevUO6LxxqDeoyUjrgjw: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9Of9YBbvsamRURogbFCTog: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9OWbkU0KtgP1fWYqA2SHuw: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O0DOmLWZKbIix2SZrBQBA: removed DONE job
23 Nov, 13:19:44 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://devel19.cnaf.infn.it:9000/9O46dhW7QO5mihD_3arqUg: removed DONE job

Look if one SandBox dir has not been cancelled:

[glite@devel19 ~]$ ls -l /var/SandboxDir/9O
total 4
drwxrwx--- 5 dteam042 glite 4096 Nov 23 13:19 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9OMsNaP2bzC3oiToyAu5hA

===>     === test PASSED === 

- Purge a job which proxy is already expired

[glite@cream-45 ~]$ ls /var/SandboxDir/Z6/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fZ6bSUBxN_5fkPNQNs73Vymog/
input  JDLOriginal  JDLStarted  output  peek  user.proxy

[glite@cream-45 ~]$ openssl x509 -noout -text -in /var/SandboxDir/Z6/https_3a_2f_2fcream-45.pd.infn.it_3a9000_2fZ6bSUBxN_5fkPNQNs73Vymog/user.proxy 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1319202904 (0x4ea17058)
        Signature Algorithm: md5WithRSAEncryption
        Issuer: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy
        Validity
            Not Before: Oct 21 13:10:04 2011 GMT
            Not After : Oct 21 13:29:04 2011 GMT
        Subject: C=IT, O=INFN, OU=Personal Certificate, L=Padova, CN=Alessio Gianelle, CN=proxy, CN=proxy
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (512 bit)
                Modulus (512 bit):
                    00:c5:b8:6e:a8:21:5d:48:da:23:10:e7:c0:13:3c:
                    97:a1:df:ef:10:27:30:fa:77:52:5f:1e:52:b8:fc:
                    74:1b:76:12:6d:63:e4:25:56:3e:6d:eb:61:13:5a:
                    a6:47:0d:bb:13:4e:b5:b2:2a:7f:22:6b:49:f9:a7:
                    b6:2c:b4:65:31
                Exponent: 65537 (0x10001)
    Signature Algorithm: md5WithRSAEncryption
        a1:82:7a:84:59:92:6b:ba:94:a6:54:a0:17:60:f6:95:31:f8:
        c0:86:b5:3b:8f:6e:3f:7e:09:80:c4:39:3b:3b:d2:f7:42:fb:
        f6:a6:fd:67:4b:fd:92:3c:23:46:f6:e1:f4:4d:60:26:71:92:
        a6:aa:b2:f5:52:eb:25:cd:25:df:11:73:20:ad:c3:79:78:b2:
        3f:19:2a:11:50:3b:43:42:9d:33:fc:36:38:19:8b:6c:ee:23:
        ce:fd:17:82:8f:01:d9:5b:14:55:ad:95:4c:d2:16:59:d5:a0:
        22:90:57:25:10:01:71:1d:ce:cf:24:4f:37:8f:65:4a:e1:9a:
        9f:50
[glite@cream-45 ~]$ date
Fri Oct 21 15:35:19 CEST 2011

[glite@cream-45 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/Z6
21 Oct, 15:35:39 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:447): https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog: removed DONE job

[ale@cream-12 UI]$ glite-wms-job-status https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog
Current Status:     Cleared 
Status Reason:      timed out, resource purge forced
Destination:        cream-22.pd.infn.it:8443/cream-lsf-creamcert1
Submitted:          Fri Oct 21 15:15:05 2011 CEST
==========================================================================

The last Clear logged event should be logged with the host proxy (not user proxy):

[ale@cream-12 WMS-service]$ glite-wms-job-logging-info --event Clear -v 2 https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog

===================== glite-wms-job-logging-info Success =====================

LOGGING INFORMATION:

Printing info for the Job : https://cream-45.pd.infn.it:9000/Z6bSUBxN_kPNQNs73Vymog
 
   ---
Event: Clear
- Arrived                    =    Fri Oct 21 15:35:39 2011 CEST
- Host                       =    cream-45.pd.infn.it
- Reason                     =    2
- Source                     =    NetworkServer
- Src instance               =    11882
- Timestamp                  =    Fri Oct 21 15:35:39 2011 CEST
- User                       =    /C=IT/O=INFN/OU=Host/L=Padova/CN=cream-45.pd.infn.it/CN=244681262
==========================================================================

===>     === test PASSED === 

- Purge a dag job 

[glite@devel19 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/8r
23 Nov, 13:28:40 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:354): https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ: 25/25 nodes removed
23 Nov, 13:28:40 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:366): https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ: removed ABORTED dag 

Check SBD of some nodes:

[glite@devel19 ~]$ ls -l /var/SandboxDir/1p/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f1pW78m9XmY-w-26zFbsoPA
ls: /var/SandboxDir/1p/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f1pW78m9XmY-w-26zFbsoPA: No such file or directory
[glite@devel19 ~]$ ls -l /var/SandboxDir/9g/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9gTb01U4t3xqz0OZYzjw6Q
ls: /var/SandboxDir/9g/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2f9gTb01U4t3xqz0OZYzjw6Q: No such file or directory

Check final status:

[ale@cream-12 UI]$ glite-wms-job-status -v 0 https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/8rdSy5Z2JdTiD4_hKVPoRQ
Current Status:     Aborted 
- Nodes information for: 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/1543Cn1u5bYvdkZhto1W_g
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/1pW78m9XmY-w-26zFbsoPA
    Current Status:     Aborted 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/9gTb01U4t3xqz0OZYzjw6Q
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/A7ElSgilrBeJG9GOyJfgEw
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/Azs0x-XOpjKeH4qr6r8Rpw
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/D5QhuhNEAN-NO_cvqDJq1g
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/DON4ZQ1vyc3G8bfeXvi-yg
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/H2CYH-muEWP7r6LQ8b2oLA
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/M2byYUQLbZcOQ_7JOrkoPA
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/R2NSChOyvZW4fegwaMGKwQ
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/RECPyDf9WM1XWhk4b2d9-A
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/SdbGoobqGLxTXoBkizcVQg
    Current Status:     Aborted 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/WvPCqVsDgPHYicPOkBjgCQ
    Current Status:     Aborted 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/by-jOy0FVhAfJrbpIRNvhA
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/c47GgReIgki2uz9PDEHfMg
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/dPva2oOwQHINZ6KFyOqYMw
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/esNNVXHVT0smIq6eg8ko0A
    Current Status:     Aborted 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/gaPt3wsKBnMCuW5C0rFUVg
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/iqRZmlOIyaDP-sZ6pTyOnQ
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/jHTFVQE8IU_tamz1S3DD-A
    Current Status:     Aborted 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/jchwYeMEgSVaEosWk9N50Q
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/k52iK4koVGkqOH2WY-EANw
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/kCw2bybHh6vzdla7UuqBMQ
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/lr4WSE3bZ4EITw0XnKdksA
    Current Status:     Cleared 
    Status info for the Job : https://devel19.cnaf.infn.it:9000/nAiA3XlVyjLTL43c9q5HIA
    Current Status:     Cleared 
==========================================================================

===>     === test PASSED ===

Published information

bdb database definitions

ldapsearch -x -H ldap://devel19.cnaf.infn.it:2170 -b 'o=grid'

# extended LDIF
#
# LDAPv3
# base <o=grid> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# grid
dn: o=grid
objectClass: organization
o: grid

# local, grid
dn: Mds-Vo-name=local,o=grid
Mds-Vo-name: local
objectClass: Mds

# resource, grid
dn: Mds-Vo-name=resource,o=grid
Mds-Vo-name: resource
objectClass: Mds

# devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid
dn: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637,M
 ds-Vo-name=resource,o=grid
GlueServiceAccessControlBaseRule: VO:atlas
GlueServiceAccessControlBaseRule: VO:alice
GlueServiceAccessControlBaseRule: VO:lhcb
GlueServiceAccessControlBaseRule: VO:cms
GlueServiceAccessControlBaseRule: VO:dteam
GlueServiceAccessControlBaseRule: VO:biomed
GlueServiceAccessControlBaseRule: VO:ops
GlueServiceAccessControlBaseRule: VO:infngrid
GlueServiceAccessControlBaseRule: VO:testers.eu-emi.eu
GlueServiceStatus: Other
GlueServiceStatusInfo: ??
objectClass: GlueTop
objectClass: GlueService
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueServiceUniqueID: devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637
GlueServiceAccessControlRule: atlas
GlueServiceAccessControlRule: alice
GlueServiceAccessControlRule: lhcb
GlueServiceAccessControlRule: cms
GlueServiceAccessControlRule: dteam
GlueServiceAccessControlRule: biomed
GlueServiceAccessControlRule: ops
GlueServiceAccessControlRule: infngrid
GlueServiceAccessControlRule: testers.eu-emi.eu
GlueServiceEndpoint: https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_serve
 r
GlueServiceVersion: 3.3.4
GlueSchemaVersionMinor: 3
GlueServiceName: INFN-CREAMTEST-WMProxy
GlueServiceType: org.glite.wms.WMProxy
GlueServiceWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl
GlueServiceSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-guid
 e.pdf
GlueForeignKey: GlueSiteUniqueID=INFN-CREAMTEST
GlueServiceOwner: atlas
GlueServiceOwner: alice
GlueServiceOwner: lhcb
GlueServiceOwner: cms
GlueServiceOwner: dteam
GlueServiceOwner: biomed
GlueServiceOwner: ops
GlueServiceOwner: infngrid
GlueServiceOwner: testers.eu-emi.eu
GlueSchemaVersionMajor: 1
GlueServiceStartTime: 2011-11-30T09:28:22+01:00

# DN, devel19.cnaf.infn.it_org.glite.wms.WMProxy_841415637, resource, grid
dn: GlueServiceDataKey=DN,GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.w
 ms.WMProxy_841415637,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: DN
GlueChunkKey: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_8
 41415637
GlueServiceDataValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel19.cnaf.infn.it

# glite-info-service_version, devel19.cnaf.infn.it_org.glite.wms.WMProxy_8414
 15637, resource, grid
dn: GlueServiceDataKey=glite-info-service_version,GlueServiceUniqueID=devel19.
 cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_version
GlueChunkKey: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_8
 41415637
GlueServiceDataValue: 1.6

# glite-info-service_hostname, devel19.cnaf.infn.it_org.glite.wms.WMProxy_841
 415637, resource, grid
dn: GlueServiceDataKey=glite-info-service_hostname,GlueServiceUniqueID=devel19
 .cnaf.infn.it_org.glite.wms.WMProxy_841415637,Mds-Vo-name=resource,o=grid
GlueSchemaVersionMajor: 1
objectClass: GlueTop
objectClass: GlueServiceData
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSchemaVersionMinor: 3
GlueServiceDataKey: glite-info-service_hostname
GlueChunkKey: GlueServiceUniqueID=devel19.cnaf.infn.it_org.glite.wms.WMProxy_8
 41415637
GlueServiceDataValue: devel19.cnaf.infn.it

# search result
search: 2
result: 0 Success

# numResponses: 8
# numEntries: 7

Stats database definitions

ldapsearch -x -H ldap://devel19.cnaf.infn.it:2170 -b 'o=infosys'

# extended LDIF
#
# LDAPv3
# base <o=infosys> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# infosys
dn: o=infosys
objectClass: organization
o: infosys

# zip, infosys
dn: CompressionType=zip,o=infosys
objectClass: CompressedContent
Hostname: localhost
CompressionType: zip
Data: file:///var/run/bdii/archive/11-11-30-11-27-58-snapshot.gz

# localhost, infosys
dn: Hostname=localhost,o=infosys
objectClass: UpdateStats
Hostname: localhost
FailedDeletes: 0
ModifiedEntries: 5
DeletedEntries: 0
UpdateTime: 0
FailedAdds: 0
FailedModifies: 0
TotalEntries: 14
QueryTime: 0
NewEntries: 1
DBUpdateTime: 0
ReadTime: 0
PluginsTime: 0
ProvidersTime: 0

# search result
search: 2
result: 0 Success

# numResponses: 4
# numEntries: 3

Stats database definitions

ldapsearch -x -H ldap://devel19.cnaf.infn.it:2170 -b 'o=glue'


# extended LDIF
#
# LDAPv3
# base <o=glue> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# glue
dn: o=glue
objectClass: organization
o: glue

# grid, glue
dn: GLUE2GroupID=grid,o=glue
objectClass: GLUE2Group
GLUE2GroupID: grid

# resource, glue
dn: GLUE2GroupID=resource,o=glue
objectClass: GLUE2Group
GLUE2GroupID: resource

# devel19.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2ServiceID=devel19.cnaf.infn.it_wmproxy_2598325561,GLUE2GroupID=resour
 ce,o=glue
GLUE2EntityValidity: 3600
GLUE2ServiceType: org.glite.wms.WMProxy
GLUE2ServiceID: devel19.cnaf.infn.it_wmproxy_2598325561
objectClass: GLUE2Service
GLUE2ServiceQualityLevel: production
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-service
GLUE2EntityOtherInfo: InfoProviderVersion=1.1
GLUE2EntityOtherInfo: InfoProviderHost=devel19.cnaf.infn.it
GLUE2ServiceComplexity: endpointType=1, share=0, resource=0
GLUE2ServiceCapability: executionmanagement.candidatesetgenerator
GLUE2ServiceCapability: executionmanagement.jobdescription
GLUE2ServiceCapability: executionmanagement.jobmanager
GLUE2EntityName: INFN-CREAMTEST-WMProxy
GLUE2ServiceAdminDomainForeignKey: INFN-CREAMTEST
GLUE2EntityCreationTime: 2011-11-30T10:29:59Z

# devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561, d
 evel19.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2EndpointID=devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMPr
 oxy_2598325561,GLUE2ServiceID=devel19.cnaf.infn.it_wmproxy_2598325561,GLUE2Gr
 oupID=resource,o=glue
GLUE2EntityValidity: 3600
GLUE2EndpointQualityLevel: production
GLUE2EndpointImplementor: gLite
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=devel19.cnaf.infn.it
GLUE2EntityOtherInfo: MiddlewareName=EMI
GLUE2EntityOtherInfo: MiddlewareVersion=1.5.0-1
GLUE2EndpointCapability: executionmanagement.candidatesetgenerator
GLUE2EndpointCapability: executionmanagement.jobdescription
GLUE2EndpointCapability: executionmanagement.jobmanager
GLUE2EndpointServiceForeignKey: devel19.cnaf.infn.it_wmproxy_2598325561
GLUE2EntityName: WMProxy endpoint for Service devel19.cnaf.infn.it_wmproxy_259
 8325561
GLUE2EndpointTechnology: webservice
GLUE2EndpointWSDL: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy.wsdl
GLUE2EndpointInterfaceName: org.glite.wms.WMProxy
GLUE2EndpointURL: https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server
GLUE2EndpointDowntimeInfo: See the GOC DB for downtimes: https://goc.egi.eu/
GLUE2EndpointImplementationVersion: 3.3.4
GLUE2EndpointSemantics: http://web.infn.it/gLiteWMS/images/WMS/Docs/wmproxy-gu
 ide.pdf
GLUE2EndpointIssuerCA: /C=IT/O=INFN/CN=INFN CA
GLUE2EndpointServingState: production
objectClass: GLUE2Endpoint
GLUE2EndpointInterfaceVersion: 1.0
GLUE2EndpointImplementationName: WMS
GLUE2EndpointTrustedCA: IGTF
GLUE2EndpointID: devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy
 _2598325561
GLUE2EndpointHealthState: ok
GLUE2EndpointStartTime: 2011-11-30T08:28:22Z
GLUE2EndpointHealthStateInfo: WMProxy httpd listening on port 7443 httpd is ru
 nning .... === WMProxy Server running instances: UID PID PPID C STIME TTY TIM
 E CMD glite 29819 29794 0 09:28 ? 00:00:01 /usr/bin/glite_wms_wmproxy_server
GLUE2EntityCreationTime: 2011-11-30T10:29:59Z

# DN_1, devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325
 561, devel19.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2ExtensionLocalID=DN_1,GLUE2EndpointID=devel19.cnaf.infn.it_wmproxy_25
 98325561_org.glite.wms.WMProxy_2598325561,GLUE2ServiceID=devel19.cnaf.infn.it
 _wmproxy_2598325561,GLUE2GroupID=resource,o=glue
GLUE2ExtensionLocalID: DN_1
GLUE2ExtensionKey: DN
objectClass: GLUE2Extension
GLUE2ExtensionValue: /C=IT/O=INFN/OU=Host/L=CNAF/CN=devel19.cnaf.infn.it
GLUE2ExtensionEntityForeignKey: devel19.cnaf.infn.it_wmproxy_2598325561_org.gl
 ite.wms.WMProxy_2598325561

# devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2598325561_Po
 licy, devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_25983255
 61, devel19.cnaf.infn.it_wmproxy_2598325561, resource, glue
dn: GLUE2PolicyID=devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProx
 y_2598325561_Policy,GLUE2EndpointID=devel19.cnaf.infn.it_wmproxy_2598325561_o
 rg.glite.wms.WMProxy_2598325561,GLUE2ServiceID=devel19.cnaf.infn.it_wmproxy_2
 598325561,GLUE2GroupID=resource,o=glue
GLUE2AccessPolicyEndpointForeignKey: devel19.cnaf.infn.it_wmproxy_2598325561_o
 rg.glite.wms.WMProxy_2598325561
GLUE2EntityValidity: 3600
objectClass: GLUE2Policy
objectClass: GLUE2AccessPolicy
GLUE2PolicyRule: VO:atlas
GLUE2PolicyRule: VO:alice
GLUE2PolicyRule: VO:lhcb
GLUE2PolicyRule: VO:cms
GLUE2PolicyRule: VO:dteam
GLUE2PolicyRule: VO:biomed
GLUE2PolicyRule: VO:ops
GLUE2PolicyRule: VO:infngrid
GLUE2PolicyRule: VO:testers.eu-emi.eu
GLUE2PolicyScheme: org.glite.standard
GLUE2EntityOtherInfo: InfoProviderName=glite-info-glue2-endpoint
GLUE2EntityOtherInfo: InfoProviderVersion=1.2
GLUE2EntityOtherInfo: InfoProviderHost=devel19.cnaf.infn.it
GLUE2PolicyUserDomainForeignKey: atlas
GLUE2PolicyUserDomainForeignKey: alice
GLUE2PolicyUserDomainForeignKey: lhcb
GLUE2PolicyUserDomainForeignKey: cms
GLUE2PolicyUserDomainForeignKey: dteam
GLUE2PolicyUserDomainForeignKey: biomed
GLUE2PolicyUserDomainForeignKey: ops
GLUE2PolicyUserDomainForeignKey: infngrid
GLUE2PolicyUserDomainForeignKey: testers.eu-emi.eu
GLUE2PolicyID: devel19.cnaf.infn.it_wmproxy_2598325561_org.glite.wms.WMProxy_2
 598325561_Policy
GLUE2EntityName: Access control rules for Endpoint devel19.cnaf.infn.it_wmprox
 y_2598325561_org.glite.wms.WMProxy_2598325561
GLUE2EntityCreationTime: 2011-11-30T10:29:59Z

# search result
search: 2
result: 0 Success

# numResponses: 8
# numEntries: 7

Performance test

Collection of 1000 nodes

Description: submission of a collection with 1000 nodes

Collection ID: https://devel19.cnaf.infn.it:9000/EFsebwUWZUWMYi0GgFrntA
Final Status:
- 1000 Done (Success)

===> test PASSED

Stress test

Description

2880 collection of 25 nodes each (total: 72000 jobs)
One collection every 60 seconds
two users
Use LCG-CEs and CREAM-CEs (with different batch systems)
Use automatic-delegation
The job is a "sleep random(666)"
Resubmission is enabled
Enable proxy renewal

Results

2880 collections submitted in 3200 seconds: 0/1/13 (min/avg/max)
Jobs correctly submitted: 72000 (85.4% to CREAM and 14.6% to LCG ces)
- DONE OK: 71600 (99.44%)
- Aborted: 400 (0.56%) (Proxy expired)

===> test PASSED

Regression Tests

Bug #81376 (glite-wms-wmproxy.restart.cron MUST support graceful wmp restart) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 81376 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 81376 


 All tests PASSED

See log file

Bug #81568 (some inconsistencies in locations for logs and configuration) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 81568 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 81568 


 All tests PASSED

See log file

Verified that all WMS log files are in this directory:

[root@cream-44 ~]# ls $WMS_LOCATION_LOG 
glite-lb-lcas.log       glite-lb-lcas.log.4.gz      glite-wms-wmproxy-purge-proxycache.log  ice.log                  wmproxy.log
glite-lb-lcas.log.1.gz  glite-lb-purger.log         glite-wms-wmproxy.restart.cron.log      jobcontoller_events.log  wmproxy_logrotate.log
glite-lb-lcas.log.2.gz  glite-lb-purger.log.1.gz    httpd-wmproxy-access.log                lcmaps.log               workload_manager_events.log
glite-lb-lcas.log.3.gz  glite-wms-purgeStorage.log  httpd-wmproxy-errors.log                logmonitor_events.log

Bug #83062 (A different template should be used for glue2 publication) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 83062 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 83062 


 All tests PASSED

See log file

Bug #85327 (glite-wms-wmproxy.restart.cron missing HOSTNAME environment variable) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 85327 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 85327 


 All tests PASSED

See log file

Bug #86485 (ICE doesn't send the iceId to CREAM) FIXED

Submitted a job through the WMS to a CREAM CE in debug mode.

Verified that in the CREAM log file the iceid is reported:

# grep selectToRetrieveJobStatusQuery /var/log/cream/glite-ce-cream.log 
20 Oct 2011 07:11:43,186 DEBUG org.glite.ce.cream.jobmanagement.db.table.JobStatusTable (JobStatusTable.java:419) - (TP-Processor25) selectToRetrieveJobStatusQuery = select job_status.type AS type,job_status.exitCode AS exitCode,job_status.failureReason AS failureReason,job_status.description AS description,job_status.time_stamp AS time_stamp,job_status.jobId AS jobId,job_status.id AS id,job.gridJobId AS gridJobId,job.workerNode AS workerNode from job_status, job where job_status.jobId = job.id and job_status.type IN ( '10', '5', '8', '7', '6', '4', '3') and job.userId = '_C_IT_O_INFN_OU_Personal_Certificate_L_Padova_CN_Massimo_Sgaravatto_dteam_Role_NULL_Capability_NULL' and job.iceId = 'C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it' and job_status.id >= 0 and time_stamp >= '2011-10-19 15:46:36.0' ORDER BY job_status.id limit 500

Verified also that in the CREAMDB the iceid is set for this job:

mysql> use creamdb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select iceId from job where id="CREAM806900601"
    -> ;
+-----------------------------------------------------+
| iceId                                               |
+-----------------------------------------------------+
| C_IT_O_INFN_OU_Host_L_Padova_CN_cream-44.pd.infn.it | 
+-----------------------------------------------------+
1 row in set (0.00 sec)

===> test PASSED

Bug #86490 (EMI WMS stops accepting jobs after 31999th submission for the same DN (on ext3)) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 86490 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 86490 


 All tests PASSED

See log file

Bug #86682 ( yaim changes ownership of /var) FIXED

[ale@cream-12 regression_tests]$ python execute_regTest.py -d 3 -b 86682 -l
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Regression TestSuite ....                         +
+++++++++++++++++++++++++++++++++++++++++++++++++++++

 Testing bug 86682 


 All tests PASSED

See log file

Bug #86769 (EMI WMS Update 6: WMS 'hangs' after several hours of operation) FIX NOT CERTIFIED

It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions. It also has passed a test done by the user who posted the bug.

Bug #86956 (Match-matching of requests with access to catalogues can cause the WM to crash) FIX NOT CERTIFIED

To use the function specified in the bug we need a jdl with DataRequirements. In any case the error is not deterministic, so you can only set this bug as hopefully fixed.

[ale@cream-12 DataReq]$ cat data-req.jdl 
###########################################
#      JDL with Data Requirements         #
###########################################

Executable = "/bin/ls";
Arguments = "-la";
StdOutput = "std.out";
StdError = "std.err";
Prologue = "prologue.sh";

FuzzyRank = true;
Environment={"LFC_HOST=prod-lfc-shared-central.cern.ch", "LFC_HOME=prod-lfc-shared-central.cern.ch:/grid/dteam/aleph"};
InputSandbox = {"prologue.sh"};
OutputSandbox = {"std.out", "std.err","prologue.out", "file.txt"};

requirements = true;

DataRequirements = {
[
DataCatalogType = "DLI";
DataCatalog ="http://prod-lfc-shared-central.cern.ch:8085/"; 
InputData = { "guid:17c667b7-9622-4c95-be68-3d15d1eacadb" };
]
};

DataAccessProtocol = "gsiftp";
AllowZippedISB=true;
RetryCount = 1;
ShallowRetryCount = 2;

[ale@cream-12 DataReq]$ glite-wms-job-list-match  -a   -c /home/ale/UI/etc/wmp_devel19.conf  data-req.jdl 

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
 The following CE(s) matching your job requirements have been found:

   *CEId*
 - cert-15.pd.infn.it:8443/cream-lsf-cert
 - cert-37.pd.infn.it:8443/cream-lsf-cert
 - cream-19.pd.infn.it:8443/cream-lsf-cert
 - cream-19.pd.infn.it:8443/cream-lsf-creamcert1
 - cream-19.pd.infn.it:8443/cream-lsf-creamcert2
 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-cert
 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest1
 - cream-26.pd.infn.it:2119/jobmanager-lcgpbs-creamtest2
 - cream-31.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-31.pd.infn.it:8443/cream-pbs-creamtest2
 - cream-38.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-38.pd.infn.it:8443/cream-pbs-creamtest2
 - prod-ce-01.pd.infn.it:8443/cream-lsf-cert
 - prod-ce-02.pd.infn.it:2119/jobmanager-lcglsf-cert
 - cream-24.pd.infn.it:8443/cream-lsf-cert
 - cream-24.pd.infn.it:8443/cream-lsf-testbedB_1
 - cream-24.pd.infn.it:8443/cream-lsf-testbedB_2
 - cream-41.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-32.pd.infn.it:8443/cream-pbs-cert
 - cream-32.pd.infn.it:8443/cream-pbs-creamtest1
 - cream-32.pd.infn.it:8443/cream-pbs-creamtest2
 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce130.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce131.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce131.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce132.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce132.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce133.cern.ch:2119/jobmanager-lcglsf-grid_2nh_dteam
 - ce133.cern.ch:2119/jobmanager-lcglsf-grid_dteam
 - ce203.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce203.cern.ch:8443/cream-lsf-grid_dteam
 - ce204.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce204.cern.ch:8443/cream-lsf-grid_dteam
 - ce205.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce205.cern.ch:8443/cream-lsf-grid_dteam
 - ce206.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce206.cern.ch:8443/cream-lsf-grid_dteam
 - ce207.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce207.cern.ch:8443/cream-lsf-grid_dteam
 - ce208.cern.ch:8443/cream-lsf-grid_2nh_dteam
 - ce208.cern.ch:8443/cream-lsf-grid_dteam
 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-cert
 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest1
 - cream-11.pd.infn.it:2119/jobmanager-lcglsf-creamtest2
 - cream-20.pd.infn.it:8443/cream-lsf-cert
 - cream-20.pd.infn.it:8443/cream-lsf-creamcert1
 - cream-20.pd.infn.it:8443/cream-lsf-creamcert2
 - cream-31.pd.infn.it:8443/cream-pbs-cert
 - cream-41.pd.infn.it:8443/cream-pbs-cert
 - cream-41.pd.infn.it:8443/cream-pbs-creamtest2

==========================================================================

Bug #87259 (WMS purger errors in jobcontroller/logmonitor) FIXED

For a job submitted to a LCG-CE, verified that in /var/log/glite/logmonitor_events.log the removing of job directory, submit file, classad file, and wrapper file is logged:

23 Nov, 15:27:44 -I- EventTerminated::process_event(): Got job terminated event for cluster 183
23 Nov, 15:27:44 -I- EventTerminated::processNormalJob(...): Job id = https://cream-44.pd.infn.it:9000/tQFtQrDkWGls48jUFr0nQw
23 Nov, 15:27:44 -I- EventTerminated::processNormalJob(...): Return value = 0
23 Nov, 15:27:44 -H- JobWrapperOutputParser::parse_file(...): Going to parse standard output file.
23 Nov, 15:27:44 -I- EventTerminated::processNormalJob(...): Real return code: 0
23 Nov, 15:27:44 -V- ProxyUnregistrar::unregister(): Unregistering user proxy...
23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing job directory: /var/jobcontrol/condorio/tQ/https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw
23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed 3 files.
23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing submit file: /var/jobcontrol/submit/tQ/Condor.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw.submit
23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed...
23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing classad file: /var/jobcontrol/submit/tQ/ClassAd.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw
23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed...
23 Nov, 15:27:44 -I- JobFilePurger::do_purge(...): Removing wrapper file: /var/jobcontrol/submit/tQ/JobWrapper.https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftQFtQrDkWGls48jUFr0nQw.sh
23 Nov, 15:27:44 -U- JobFilePurger::do_purge(...): Removed...
23 Nov, 15:27:44 -I- MonitorLoop::run(): Spent 0 seconds in the last file loop.
23 Nov, 15:27:44 -I- MonitorLoop::run(): Must wait for other 6 seconds.
23 Nov, 15:27:50 -I- MonitorLoop::run(): No new event found, going to sleep.
23 Nov, 15:27:50 -I- MonitorLoop::run(): Checking each 5 seconds for new events.

Verified that they have been actually removed:


[root@cream-44 ~]# ls /var/jobcontrol/condorio/tQ/
[root@cream-44 ~]# ls /var/jobcontrol/submit/tQ/
[root@cream-44 ~]#

Submitted a job:

[ale@cream-12 UI]$ glite-wms-job-submit -a   -c /home/ale/UI/etc/wmp_cream-44.conf test.jdl

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA

==========================================================================


[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        cream-40.pd.infn.it:8443/cream-pbs-creamtest2
Submitted:          Wed Nov 23 15:42:23 2011 CET
==========================================================================

Copied the sandbox dir:

[root@cream-44 uX]# cd /var/SandboxDir/tY/
[root@cream-44 tY]# ls -l
total 4
drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA
[root@cream-44 tY]# cp -r -p https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA/ https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA.old
[root@cream-44 tY]# ls -l
total 8
drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA
drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA.old

Retrieved the OSB when the job is done:

[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job Terminated Successfully
Destination:        cream-40.pd.infn.it:8443/cream-pbs-creamtest2
Submitted:          Wed Nov 23 15:42:23 2011 CET
==========================================================================

[ale@cream-12 UI]$ glite-wms-job-output https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA

Connecting to the service https://cream-44.pd.infn.it:7443/glite_wms_wmproxy_server

================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/ale_tYM93krTdeUb7BX5qJoLDA

================================================================================


[ale@cream-12 UI]$ cat /tmp/jobOutput/ale_tYM93krTdeUb7BX5qJoLDA/out.log 
total 12
drwxr-xr-x 2 dteam009 dteam 4096 Nov 23 15:42 .
drwxr-xr-x 3 dteam009 dteam 4096 Nov 23 15:42 ..
-rw-r--r-- 1 dteam009 dteam  367 Nov 23 15:42 .BrokerInfo
-rw-r--r-- 1 dteam009 dteam    0 Nov 23 15:42 out.log
-rw-r--r-- 1 dteam009 dteam    0 Nov 23 15:42 .tmp_file
[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        cream-40.pd.infn.it:8443/cream-pbs-creamtest2
Submitted:          Wed Nov 23 15:42:23 2011 CET
==========================================================================

Restored the sandbox dir in the WMS:

[root@cream-44 tY]# mv https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA.old/ https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA
[root@cream-44 tY]# ls -l
total 4
drwxrwx--- 5 dteam009 glite 4096 Nov 23 15:42 https_3a_2f_2fcream-44.pd.infn.it_3a9000_2ftYM93krTdeUb7BX5qJoLDA

Purged the job from LBProxy and from LBServer:

[root@cream-44 ~]# echo "https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA" > jobid.txt
[root@cream-44 ~]#  /usr/bin/glite-lb-purge -m cream-44.pd.infn.it -x -j jobid.txt 
The jobs were not dumped.
[root@cream-44 ~]# /usr/bin/glite-lb-purge -m cream-44.pd.infn.it -j jobid.txt 
The jobs were not dumped.

Verified that job status returns Identifier removed (i.e. error EIDRM):

[ale@cream-12 UI]$ glite-wms-job-status https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA



**** Error: API_NATIVE_ERROR ****  
Error while calling the "Job:getStatus" native api 
Unable to retrieve the status for: https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA
glite.lb.Exception: edg_wll_JobStatus: Identifier removed: matching job already purged
   at glite::lb::Job::status[./src/Job.cpp:87]

On the WMS, purged the sandboxdir, as user glite, invoking the command glite-wms-purgeStorage.sh and verifying that Identifier removed (i.e. error 43) is returned:

[glite@cream-44 ~]$ /usr/sbin/glite-wms-purgeStorage.sh -p /var/SandboxDir/tY
23 Nov, 15:49:35 -E: [Error] query_job_status(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:134): https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA: edg_wll_JobStat [43] Identifier removed(matching job already purged)
23 Nov, 15:49:35 -I: [Info] operator()(/builddir/build/BUILD/glite-wms-purger-3.3.2/build/../src/purger.cpp:306): https://cream-44.pd.infn.it:9000/tYM93krTdeUb7BX5qJoLDA: forced removal, unknown/removed L&B job

Verified that the sandboxdir has been removed:

[root@cream-44 ~]# ls -l /var/SandboxDir/tY
total 0
[root@cream-44 ~]#

===> test PASSED

Bug #87994 yaim-wms doesn't fill the attributes get_acbr and get_owner in the conf file for Glue2 FIXED

Check the attributes get_acbr and get_owner in /etc/glite/info/service/glite-info-glue2-wmproxy.conf

[root@cream-44 ~]# grep get_acbr /etc/glite/info/service/glite-info-glue2-wmproxy.conf
get_acbr = echo -e "VO:alice\nVO:atlas\nVO:cms\nVO:lhcb\nVO:dteam\nVO:testers.eu-emi.eu"
[root@cream-44 ~]# grep get_owner /etc/glite/info/service/glite-info-glue2-wmproxy.conf
get_owner = echo -e "alice\natlas\ncms\nlhcb\ndteam\ntesters.eu-emi.eu"

===> test PASSED

Bug #88072 EMI-1 WMS cannot submit to ARC CEs FIXED

-bash-3.2$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ

======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted:          Mon Nov 21 16:01:15 2011 CET
==========================================================================



-bash-3.2$ glite-wms-job-output https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server

================================================================================

                       JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ

================================================================================



-bash-3.2$ cat /tmp/jobOutput/traldi_pl6ZSNy7MNWNSH6x8650DQ/out.txt
total 32
drwx------ 2 cmsuser43 cmsuser 4096 Nov 21 17:02 .
drwx------ 3 cmsuser43 cmsuser 4096 Nov 21 17:02 ..
-rw------- 1 cmsuser43 cmsuser  118 Nov 21 17:02 https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fpl6ZSNy7MNWNSH6x8650DQ.output
-rw-r--r-- 1 cmsuser43 cmsuser    0 Nov 21 17:02 out.txt
-rw-r--r-- 1 cmsuser43 cmsuser 1359 Nov 21 17:02 pippo.jdl
-rw------- 1 cmsuser43 cmsuser    2 Nov 21 17:02 tmp.bUujW32646
-rw------- 1 cmsuser43 cmsuser  103 Nov 21 17:02 tmp.cNkaG32632
-rw------- 1 cmsuser43 cmsuser  103 Nov 21 17:02 tmp.exhlZ32647
-rw------- 1 cmsuser43 cmsuser    2 Nov 21 17:02 tmp.gZIii32631
-rw------- 1 cmsuser43 cmsuser    0 Nov 21 17:02 tmp.SdwGF32645 

[ale@cream-12 WMS-service]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ 


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/pl6ZSNy7MNWNSH6x8650DQ
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        jade-cms.hip.fi:2811/nordugrid-GE-arc
Submitted:          Mon Nov 21 16:01:15 2011 CET
==========================================================================

===> test PASSED

Bug #88096 CA certificate verification failed FIXED NOT CERTIFIED

It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions.

Bug #88097 Collection not terminated FIXED NOT CERTIFIED

It is not easy to verify this bug, but the WMS has passed some stress test of one or two days of submissions.

Bug #88450 Ice configuration section doesn't use environment variable FIXED

[root@cream-44 ~]# cat /etc/glite-wms/glite_wms.conf | grep -i " logfile"
    logfile   =   "${WMS_LOCATION_LOG}/ice.log";
    LogFile  =  "${WMS_LOCATION_LOG}/wmproxy.log";
    LogFile  =  "${WMS_LOCATION_LOG}/networkserver_events.log";
    LogFile   =  "${WMS_LOCATION_LOG}/workload_manager_events.log";
    LogFile  =  "${WMS_LOCATION_LOG}/logmonitor_events.log";
    LogFile  =  "${WMS_LOCATION_LOG}/jobcontoller_events.log";

===> test PASSED

Bug #88558 WM creates submit requests with old sequence code upon a replan FIXED

Event: Resubmission
- Arrived                    =    Wed Nov 23 19:16:16 2011 CET
- Host                       =    devel19.cnaf.infn.it
- Level                      =    SYSTEM
- Priority                   =    synchronous
- Reason                     =    token still exists
- Result                     =    SHALLOW
- Seqcode                    =    UI=000000:NS=0000000004:WM=000007:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source                     =    WorkloadManager
- Src instance               =    13159
- Tag                        =    job will be replanned
- Timestamp                  =    Wed Nov 23 19:16:16 2011 CET
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy

[...]

Event: EnQueued
- Arrived                    =    Wed Nov 23 19:16:16 2011 CET
- Host                       =    devel19.cnaf.infn.it
- Level                      =    SYSTEM
- Priority                   =    synchronous
- Queue                      =    /var/workload_manager/jobdir
- Result                     =    OK
- Seqcode                    =    UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000
- Source                     =    WorkloadManager
- Src instance               =    13159
- Timestamp                  =    Wed Nov 23 19:16:16 2011 CET
- User                       =    /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Job            =   
      
       [
        version = "1.0.0"; 
        arguments = 
         [
          ad = 
           [
            NodeName = "Node_jdl_953"; 
            FuzzyRank = true; 
            CEInfoHostName = "cream-31.pd.infn.it"; 
            OutputSandboxPath = "/var/SandboxDir/ji/https_3a_2f_2fdevel19.cnaf.infn.it_3a9000_2fjiHL26jdUVDO7Bl6BJlhEQ/output"; 
            StdOutput = "out.log"; 
            GlobusResourceContactString = "cream-31.pd.infn.it:8443/cream-pbs"; 
            QueueName = "creamtest2"; 
            ShallowRetryCount = 3; 
            RetryCount = 2; 
            Executable = "/bin/sleep"; 
            Type = "job"; 
            LB_sequence_code = "UI=000000:NS=0000000004:WM=000009:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000";

===> test PASSED

Bug #88569 WMS: LCMAPS should log on file only and not on syslog FIXED

Check that no messages about LCMAPS have been logged by wmproxy on syslog:

[root@cream-44 ~]# grep glite_wms_wmproxy_server /var/log/messages* | grep -i "LCMAPS"
[root@cream-44 ~]#

And also that there is a log file for lcmaps:

[root@cream-44 ~]# ls -l /var/log/glite/lcmaps.log 
-rw-r--r-- 1 glite glite 659344 Nov 21 15:43 /var/log/glite/lcmaps.log

===> test PASSED

Nagios probes test

Clean installation

Environment

Clean SL5 installation plus an EMI-UI according to EMI guidelines.

[root@devel15 ~]# cat /etc/emi-version
1.6.0-1

Repositories

wget --no-check-certificate https://twiki.cern.ch/twiki/pub/EMI/EMI1Updates/testing.repo -O /etc/yum.repos.d/testing.repo
wget http://repository.egi.eu/sw/production/sam/1/repofiles/sam.repo -O /etc/yum.repos.d/sam.repo

Install

yum install emi-wms-nagios

[root@devel15 ~]# yum install emi-wms-nagios
Loaded plugins: downloadonly, kernel-module, priorities, protect-packages, protectbase, security, verify, versionlock
Repository EGI-trustanchors is listed more than once in the configuration
EMI-1-base                                                                                                                            | 1.9 kB     00:00     
EMI-1-testing-base                                                                                                                    | 1.9 kB     00:00     
EMI-1-third-party                                                                                                                     | 1.9 kB     00:00     
EMI-1-updates                                                                                                                         | 1.9 kB     00:00     
Oracle                                                                                                                                | 1.9 kB     00:00     
core                                                                                                                                  | 1.1 kB     00:00     
egi-sam                                                                                                                               | 1.9 kB     00:00     
extras                                                                                                                                | 3.7 kB     00:00     
4168 packages excluded due to repository priority protections
23 packages excluded due to repository protections
Reading version lock configuration
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package emi-wms-nagios.noarch 0:1.0.0-1.el5 set to be updated
--> Processing Dependency: nagios-submit-conf >= 0.2 for package: emi-wms-nagios
--> Processing Dependency: python-GridMon >= 1.1.10 for package: emi-wms-nagios
--> Processing Dependency: python-ldap for package: emi-wms-nagios
--> Running transaction check
---> Package nagios-submit-conf.noarch 0:0.2-2.el5 set to be updated
---> Package python-GridMon.noarch 0:1.1.12-1.el5 set to be updated
---> Package python-ldap.x86_64 0:2.2.0-2.1 set to be updated
--> Finished Dependency Resolution
Beginning Kernel Module Plugin
Finished Kernel Module Plugin

Dependencies Resolved

=============================================================================================================================================================
 Package                                   Arch                          Version                             Repository                                 Size
=============================================================================================================================================================
Installing:
 emi-wms-nagios                            noarch                        1.0.0-1.el5                         EMI-1-testing-base                         61 k
Installing for dependencies:
 nagios-submit-conf                        noarch                        0.2-2.el5                           egi-sam                                   3.4 k
 python-GridMon                            noarch                        1.1.12-1.el5                        egi-sam                                   153 k
 python-ldap                               x86_64                        2.2.0-2.1                           sl-base                                   123 k

Transaction Summary
=============================================================================================================================================================
Install       4 Package(s)
Upgrade       0 Package(s)

Total download size: 340 k
Is this ok [y/N]: y
Downloading Packages:
(1/4): nagios-submit-conf-0.2-2.el5.noarch.rpm                                                                                        | 3.4 kB     00:00     
(2/4): emi-wms-nagios-1.0.0-1.el5.noarch.rpm                                                                                          |  61 kB     00:00     
(3/4): python-ldap-2.2.0-2.1.x86_64.rpm                                                                                               | 123 kB     00:00     
(4/4): python-GridMon-1.1.12-1.el5.noarch.rpm                                                                                         | 153 kB     00:00     
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                         85 kB/s | 340 kB     00:04     
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : python-ldap                                                                                                                           1/4 
  Installing     : nagios-submit-conf                                                                                                                    2/4 
  Installing     : python-GridMon                                                                                                                        3/4 
  Installing     : emi-wms-nagios                                                                                                                        4/4 

Installed:
  emi-wms-nagios.noarch 0:1.0.0-1.el5                                                                                                                        

Dependency Installed:
  nagios-submit-conf.noarch 0:0.2-2.el5                 python-GridMon.noarch 0:1.1.12-1.el5                 python-ldap.x86_64 0:2.2.0-2.1                

Complete!

Upgrade from production

N/A: This is the initial release. No previous version to upgrade from.

Unit Tests

N/A: The nagios probes are interpretted python scripts. Unit tests do not apply.

Regression Tests

N/A: This is the initial release. No associated bugs exist.

Functional Tests

Preparation

Log as a user and create a valid voms proxy

[root@devel15 ~]# su - gianelle
[gianelle@devel15 ~]$ voms-proxy-init -voms dteam
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle
Creating temporary proxy ....................... Done
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ............................................. Done

Your proxy is valid until Sat Nov 26 01:46:46 2011

WMSProbe

State + Monit

[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobState
OK: 
OK: 
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL


Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA

The job identifier has been saved in the following file:
/var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobID

==========================================================================


[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: [Scheduled] https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA
OK: [Scheduled] https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA
glite-wms-job-status https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        devce.cnaf.infn.it:8443/cream-pbs-cert
Submitted:          Fri Nov 25 13:53:47 2011 CET
==========================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
[Scheduled] : 1|jobs_processed=1;; DONE=0;; RUNNING=0;; SCHEDULED=1;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2
[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobStateOK: Active job - Scheduled [2011-11-25T12:54:11Z]
OK: Active job - Scheduled [2011-11-25T12:54:11Z]
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL
Active job - Scheduled [2011-11-25T12:54:11Z]
[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobSubmit-dteam>
OK: success.

glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA 2>&1

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - option --nosubdir specified: 
output files with same name will be overridden


Warning - Directory already exists: 
/var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA
have been successfully retrieved and stored in the directory:
/var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput

================================================================================
metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: success.

glite-wms-job-output --noint --nosubdir --dir /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA 2>&1

Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


Warning - option --nosubdir specified: 
output files with same name will be overridden


Warning - Directory already exists: 
/var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA
have been successfully retrieved and stored in the directory:
/var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput

================================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
Done : 1|jobs_processed=1;; DONE=1;; RUNNING=0;; SCHEDULED=0;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2
[gianelle@devel15 ~]$  glite-wms-job-status https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/YQCH-0vFjnHyPTKVfY9EeA
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        devce.cnaf.infn.it:8443/cream-pbs-cert
Submitted:          Fri Nov 25 13:53:47 2011 CET
==========================================================================

[gianelle@devel15 ~]$ cat /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobOutput/gridjob.out 
vgrid01.cnaf.infn.it

Summary: PASSED

State + Monit + Cancel

[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobStateOK: 
OK: 
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL


Connecting to the service https://devel19.cnaf.infn.it:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA

The job identifier has been saved in the following file:
/var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobID

==========================================================================


[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobMonit --pass-check-dest active
metric results >>> <devel19.cnaf.infn.it,emi.wms.WMS-JobState-dteam>
OK: [Scheduled] https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA
OK: [Scheduled] https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA
glite-wms-job-status https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA
Current Status:     Scheduled 
Status Reason:      unavailable
Destination:        cremino.cnaf.infn.it:8443/cream-pbs-cloudtf
Submitted:          Fri Nov 25 13:56:16 2011 CET
==========================================================================
OK: Jobs processed - 1
OK: Jobs processed - 1
[Scheduled] : 1|jobs_processed=1;; DONE=0;; RUNNING=0;; SCHEDULED=1;; SUBMITTED=0;; READY=0;; WAITING=0;; ABORTED=0;; CANCELLED=0;; CLEARED=0;; MISSED=0;; UNDETERMINED=0;; unknown=0;1;2
[gianelle@devel15 ~]$ /usr/libexec/grid-monitoring/probes/emi.wms/WMS-probe --vo dteam -x /tmp/x509up_u502 -H devel19.cnaf.infn.it -m emi.wms.WMS-JobCancel
OK: job cancelled
OK: job cancelled
Testing from: devel15.cnaf.infn.it
DN: /C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=Alessio Gianelle/CN=proxy
VOMS FQANs: /dteam/Role=NULL/Capability=NULL, /dteam/NGI_IT/Role=NULL/Capability=NULL
Job cancellation request sent:
glite-wms-job-cancel --noint  -i /var/lib/gridprobes/dteam/emi.wms/WMS/devel19.cnaf.infn.it/jobID
Job bookkeeping files deleted.
[gianelle@devel15 ~]$ glite-wms-job-status https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA


======================= glite-wms-job-status Success =====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://devel19.cnaf.infn.it:9000/WwIF0DsmFHJsdqPYE-VEfA
Current Status:     Cancelled 
Destination:        cremino.cnaf.infn.it:8443/cream-pbs-cloudtf
Submitted:          Fri Nov 25 13:56:16 2011 CET
==========================================================================

Summary: PASSED

Attachments

Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
log	81376.log	manage	5.1 K	2011-11-23 - 15:30	AlessioGianelle
log	81568.log	manage	5.7 K	2011-11-23 - 15:31	AlessioGianelle
log	83062.log	manage	14.0 K	2011-11-23 - 15:32	AlessioGianelle
log	85327.log	manage	5.8 K	2011-11-23 - 15:31	AlessioGianelle
log	86490.log	manage	171.6 K	2011-11-23 - 15:32	AlessioGianelle
log	86682.log	manage	5.4 K	2011-11-23 - 15:31	AlessioGianelle
txt	P2284_Certification_Report.txt	manage	3.0 K	2011-12-05 - 13:01	AlessioGianelle
log	WMSService-TS_20111020163927.log	manage	68.1 K	2011-10-20 - 15:24	AlessioGianelle
log	WMSService-TS_20111117120628.log	manage	24.0 K	2011-11-22 - 10:51	AlessioGianelle
log	WMSService-TS_20111122154717.log	manage	27.5 K	2011-11-22 - 15:00	AlessioGianelle
log	WMSService-TS_20111123115945.log	manage	58.1 K	2011-11-23 - 11:03	AlessioGianelle
log	WMSService-TS_20111123122227.log	manage	41.8 K	2011-11-23 - 11:30	AlessioGianelle
log	WMSService-TS_20111123125702.log	manage	208.0 K	2011-11-23 - 12:35	AlessioGianelle
tgz	cancel.tgz	manage	15.5 K	2011-11-22 - 10:52	AlessioGianelle
tgz	collection.tgz	manage	6.7 K	2011-11-22 - 10:52	AlessioGianelle
txt	configure.txt	manage	44.8 K	2011-12-02 - 09:58	AlessioGianelle	Configure
tgz	dag.tgz	manage	4.0 K	2011-11-22 - 10:53	AlessioGianelle
tgz	delegation.tgz	manage	4.1 K	2011-11-22 - 10:53	AlessioGianelle
tgz	epilogue.tgz	manage	6.8 K	2011-11-22 - 10:54	AlessioGianelle
txt	install.txt	manage	168.4 K	2011-12-02 - 09:58	AlessioGianelle
tgz	listmatch.tgz	manage	7.5 K	2011-11-22 - 10:54	AlessioGianelle
tgz	parallel.tgz	manage	4.6 K	2011-11-22 - 10:54	AlessioGianelle
tgz	parametric.tgz	manage	7.8 K	2011-11-22 - 10:55	AlessioGianelle
tgz	perusal.tgz	manage	8.8 K	2011-11-22 - 10:55	AlessioGianelle
tgz	pr.tgz	manage	19.4 K	2011-11-22 - 10:55	AlessioGianelle
tgz	resubmit.tgz	manage	6.5 K	2011-11-22 - 10:57	AlessioGianelle
tgz	submitcoll.tgz	manage	5.9 K	2011-11-22 - 10:56	AlessioGianelle
tgz	submitnormal.tgz	manage	5.5 K	2011-11-22 - 10:56	AlessioGianelle
txt	update.txt	manage	72.6 K	2011-12-02 - 15:52	AlessioGianelle

Topic revision: r68 - 2012-03-19 - AlessioGianelle

Account
- Log In

Edit
Attach