This is a static archive of the previous Open Grid Forum GridForge content management system saved from host forge.ogf.org file /sf/wiki/do/viewPage/projects.glue-wg/wiki/WLCGRequirementsFor2.0ToCoverResourceInstalledCapacityUseCases at Thu, 03 Nov 2022 00:05:53 GMT SourceForge : View Wiki Page: WLCGRequirementsFor2.0ToCoverResourceInstalledCapacityUseCases

Project Home

Tracker

Documents

Tasks

Source Code

Discussions

File Releases

Wiki

Project Admin
Search Wiki Pages Project: GLUE     Wiki > WLCGRequirementsFor2.0ToCoverResourceInstalledCapacityUseCases > View Wiki Page
wiki2075: WLCGRequirementsFor2.0ToCoverResourceInstalledCapacityUseCases

Use cases that Glue 2.0 should cover

General note: it is common agreement that information should be collected as dynamically as possible and not left to the site admin to provide.

1. Publishing information to provide the WLCG management with a view of the "total installed capacity" at a site. This includes resources that have not yet been configured/assigned to a VO or resources that are temporary down or offline. Measurements are not required to be 100% accurate or 100% reliable. Daily reports can be used by the cesga portal in order to come up with the monthly reports requested by the management.

In order to satisfy this requirement in Glue 1.3 we have foreseen to consider the following:

  • Computing Resources
    • A description of the hardware used at a site for a specific VO. Such description includes:
      • A static description of the Subcluster in terms of Benchmark of a representative CPU, number of cores and CPUs in a cluster, share per VO of the resources of a Cluster. Such description includes nodes that are in state down or offline. Subclusters can be heterogeneous and therefore a reference CPU is also published for accounting purposes and to allow VOs to crrectly select the resources able to execute a specific job.
      • A dynamic description of resource usage total and per VO.
* Storage Resources - A description of the disk and tape resources used at a site for a specific VO. Such description includes: * A static description of the disk and tape resources represented by the InstalledOnlineCapacity Capability of a Storage Area. Here even resources that are offline (but known to the system) can be described. * A dynamic description of the disk and tape resources available to a VO: -- ReservedOnlineSize -- TotalOnlineSize -- UsedOnlineSize * Specific attributes of storage areas are published as well: -- By default a storage area is readeable -- The storage area is writeable if there is a VOInfo object associated publishing a Path -- Other attributes of a storage area are: + scratch + stage

2. Publishing information to provide the WLCG and VO management with a view of the resource assignment per VO at sites. This includes only resources that have been configured and explicitly assigned to a VO. An exception is made for shared resources. In this case it is accepted to provide only aggregated information for the common usage. Measurements are not required to be 100% accurate or 100% reliable. Daily reports can be used by the cesga portal in order to come up with the monthly reports requested by the management.

3. Publishing information to allow VO operators to monitor the VO usage of the resources. This is specifically important for monitoring the usage of storage staging areas or to monitor the usage of disk only areas.

4. Allowing clients to select resources based on their advertised characteristics and ensure that at least some agreed set of the advertised characteristics have a clear meaning. The selection process MUST be deterministic, at least for some specific and agreed selection algorithms.

- In particular there is the need of not associating the SE Unique ID to a specific hostname. In fact, an SE can have multiple control protocols: SRM, xroot, etc. that can require several host names. Furthermore, an SE can have multiple SRM endpoints, each one serving a specific VO. Xroot installations normally do not share the same resources with SRM but they can (in USA there are cases). Xroot resources will be published as one unique SA that will publish both disk and tapes. No need to publish specific characteristics such as stage, scratch, etc since all this is hidden behind the xroot protocol management. - Both the WMS and GFAL/lcg-utils use/make assumption on the SE Unique ID for instance to find the CE close to a specific SE or to copy and register an output data file on a specific SE.

In what follows we list the queries performed by the WLCG FTS and lcg-utils/gfal clients in order to select storage resources. Furthermore, the queries and algorithms used by the WLCG WMS are also listed. 1. Get SRM type and endpoint given the SE Unique ID. - Output attributes: GlueSEPort, GlueSEStatus (not used for the moment), GlueServiceVersion, GlueServiceType, GlueServiceEndpoint

- Request: * " (| (GlueSEUniqueID=<SRMHOST-FQDN>) (& (GlueServiceType=srm*) (GlueServiceEndpoint=*://<SRM-FQDN>:<PORT>*)))"

2. Get VOInfo object given the SE Unique ID. - Output attributes: GlueVOInfoPath, GlueChunkKey (for possible GlueSALocalID)

- Request: * "(& <ACBRs> (GlueVOInfoTag=<spacetokendesc>) (GlueChunkKey=GlueSEUniqueID=<SRMHOST-FQDN>))" * "(& <ACBRs> (!(GlueVOInfoTag=*)) (GlueChunkKey=GlueSEUniqueID=<SRMHOST-FQDN>))" (if no spacetoken)

3. Get SA object (if no path in VOInfo, or no VOInfo) given SA Local ID and SE Unique ID. - Output attributes: GlueSAPath, GlueSARoot (for backward compatibility) - Request: * "(& (GlueSALocalID=<SALocalID_or_VO>) (GlueChunkKey=GlueSEUniqueID=<SRMHOST-FQDN>))"

We are waiting for the answer of the developers to list the WMS case.

 




The Open Grid Forum Contact Webmaster | Report a problem | GridForge Help
This is a static archive of the previous Open Grid Forum GridForge content management system saved from host forge.ogf.org file /sf/wiki/do/viewPage/projects.glue-wg/wiki/WLCGRequirementsFor2.0ToCoverResourceInstalledCapacityUseCases at Thu, 03 Nov 2022 00:05:53 GMT