Inter-dependent multi-site Job Submissions

Keywords

Dependency, multi-site, job groups, hierarchical

Goal and description

There is a need to allow and provide the ability for users to submit jobs to more than one site in order to accomplish the user's main goal. For example, the user wants to simulate a local weather model (e.g. for Germany) based on area weather data (e.g. EU mid-range weather data), and visualise the data for TV news broadcasting in Weather Forecast shows. Because the sites involved offer various degrees of computing power, and the user has entered different agreements with each site, the user selects

1. To pick the data from ECMWF (European Centre for Mid Range Weather Forecast),
2. To simulate weather conditions for Germany at the DWD (Deutscher Wetterdienst, German Weather Service), and
3. To render the results as MPEG movies for temperature distribution, cloud formations, etc. at FZJ (Research Centre Juelich) in Germany.

Actors/stakeholders

User

The user is the main creator of the workflow that is submitted to a UNICORE Grid.

Site EMS

The site's EMSs receive submitted workflows and manage their enactment and execution. EMSs MAY alter the submitted workflow if the alteration does not violate the original semantics of the submitted workflow (e.g. EMSs may replace idiomatic job patterns with optimised replacements that guarantee the same outcome).

Assumptions

Preconditions

Sites that are involved in such interaction need at least a basic agreement about mutual resource usage and accounting, fairly similar to peering agreements between Internet backbone providers. Such agreements exist in UNICORE Grids.

Main flow of events (Basic, alternate, exceptional)

Postconditions

Success requirements

Special Requirements

Issues

• Job groups or grouping is not terminally necessary for workflows. However, job groups that can recursively nest in each other add a great value of convenience to Job Management, Workflow management and design, and site execution management optimisation.

Sources/references

Please visit the UNICORE website for more information and details.

OGF sponsor/stakeholder/interested party