Projects

• Flux core includes all the main components for Flux Framework.

  

  flux-core
  project
  flux-core

  flux-framework/flux-core

• The Fluxion scheduler

  

  flux-sched
  project
  flux-sched

  flux-framework/flux-sched

• By default, a Flux system instance treats users equally and schedules work based on demand, without consideration of a user’s history of resource consumption, or what share of available resources their organization considers they should be entitled to use relative to other competing users. Flux-accounting adds a database which stores site policy, banks with with user/project associations, and metrics representing historical usage.

  

  flux-accounting
  project
  flux-accounting

  flux-framework/flux-accouting

• Flux PMIX description

  

  flux-pmix
  project
  flux-pmix

  flux-framework/flux-pmix

• Flux PMIX description

  

  flux-security
  project
  flux-security

  flux-framework/flux-security

• The Flux Operator will allow you to define a spec for a persistent or one-off "MiniCluster" - an entire setup of Flux Framework running in a set of networked pods in Kubernetes.

  

  flux-operator
  project
  flux-operator

  flux-framework/flux-operator

• The Flux Operator will allow you to define a spec for a persistent or one-off "MiniCluster" - an entire setup of Flux Framework running in a set of networked pods in Kubernetes.

  

  flux-operator
  project
  flux-operator

  flux-framework/flux-operator

• The Flux RESTful API can be used to programmatically interact with Flux, via REST. As an example, the Flux Operator can deploy the API to your cluster, and users can submit jobs to Kubernetes programatically using it. It is not limited to Kubernetes, however. The API can be provided alongside a Flux installation to give a different kind of programmatic access to user scripts. Additionally, it provides authentication and an OAauth2 style flow if desired.

  

  flux-restful-api
  project
  flux-restful-api

  flux-framework/flux-restful-api

• Custom Python bindings to install alongside Flux. The version you install should coincide with the version of Flux Core installed.

  

  flux-python
  project
  flux-python

  flux-framework/flux-python

• DYAD: DYnamic and Asynchronous Data Streamliner

  

  dyad
  project
  dyad

  flux-framework/dyad

Components

• A module is a core service that is typically run directly by flux core or another primarily component. Modules are grouped by service types.

  

  module
  component
  module

• A plugin is an extension to a service module. Plugins are grouped by topics or functionality.

  

  plugin
  component
  plugin

• Each broker module runs in its own thread as part of the broker executable, communicating with other components using messages. Broker modules are dynamically loadable with the flux-module(1) command. Core services like the KVS, job manager, and scheduler are implemented using broker modules

  

  broker module
  component
  broker module

• The job manager orchestrates a job’s life cycle. Jobtap plugins extend the job manager, arranging for callbacks at different points in the job life cycle. Jobtap plugins may be dynamically loaded with the flux-jobtap(1) command. An example of a jobtap plugin is the Flux accounting multi-factor priority plugin, which updates a job’s priority value when it enters the PRIORITY state.

  

  jobtap plugin
  component
  jobtap plugin

• When a job is started, the flux-shell(1) is the process parent of job tasks on each node. Shell plugins extend the job environment and can be configured on a per-job basis using the --setopt option of flux-run(1) and related job submission commands. affinity, pmi, and pty are examples of Flux shell plugins.

  

  shell plugin
  component
  shell plugin

• Flux commands open a connection to a particular Flux instance by specifying a URI. The scheme portion of the URI may refer to a native connection method such as local or ssh. Native connection methods are implemented as plugins called connectors. See flux_open(3).

  

  connector plugin
  component
  connector plugin

• Other URI schemes must be resolved to a native form before they can be used. Resolvers for new schemes may be added as plugins. For example, the lsf resolver plugin enables LSF users to connect to Flux instances running as LSF jobs by specifying a lsf:JOBID URI. See flux-uri(1).

  

  uri resolver plugin
  component
  uri resolver plugin

• Jobs may be rejected at ingest if their jobspec fails one of a set of configured validator plugins. The basic validator ensures the jobspec conforms to the jobspec specification. The feasibility plugin rejects job that the scheduler determines would be unable to run given the instance’s resource set. The require-instance plugin rejects jobs that do not run in a new Flux instance. See flux-config-ingest(5).

  

  validator plugin
  component
  validator plugin

• The frobnicator allows a set of configured plugins to modify jobspec at submission time. For example the defaults plugin sets configured default values for jobspec attributes such as duration and queue. See flux-config-ingest(5).

  

  frobnicator plugin
  component
  frobnicator plugin

Modules

• The Flux Key Value Store (KVS) implements hierarchical key namespaces layered atop the content storage service. Namespaces are organized as hash trees of content-addressed tree objects and values. Generally speaking, the KVS stores metadata about jobs.

  

  Key Value Store (KVS)
  module
  Key Value Store (KVS)

  flux-core

• The Resource module is backed by a resource model that provides a conceptual model for resources described and managed by Flux. (But what does it do?)

  

  Resource
  module
  Resource

  flux-core

• A distributed barrier service, which servers as some kind of block until a set of conditions (resources?) are achieved.

  

  Barrier
  module
  Barrier

  flux-core

• Content addressable storage with files back end. There are four operations provided (e.g., to get, put, load, and store) and these content operations are the main storage behind the Flux KVS.

  

  Content Files
  module
  Content Files

  flux-core

• A simple module variant of the scheduler, ideal for testing cases (more detail needed here).

  

  Sched Simple
  module
  Sched Simple

  flux-core

• This broker module is different from others in that it doens't provide an RPC service endpoint, but rather opens a local domain socket for flux_open. When you connect to a local:// URI you are using this local domain socket. Plugins for connectors are in a different location in flux-core under src/connectors, while modules are under src/modules.

  

  Connector Local
  module
  Connector Local

  flux-core

• This appears to be a service that responds to systemd IPC calls? https://0pointer.net/blog/the-new-sd-bus-api-of-systemd.html, How is this used in Flux?

  

  Sdbus
  module
  Sdbus

  flux-core

• This service helps to run subprocesses under systemd as transient units. (Why do we want to do that?)

  

  Sdexec
  module
  Sdexec

  flux-core

• Akin to content files, this is content addressable storage with an S3 back end. There are four operations provided (e.g., to get, put, load, and store) and these content operations are the main storage behind the Flux KVS.

  

  Content S3
  module
  Content S3

  flux-core

• Akin to content files, this is content addressable storage with an sqlite backend There are four operations provided (e.g., to get, put, load, and store) and these content operations are the main storage behind the Flux KVS.

  

  Content Sqlite
  module
  Content Sqlite

  flux-core

• The Flux cron service offers an interface for executing commands on triggers such as a time interval or Flux events. The service is implemented as a Flux extension module which, when loaded, manages a set of cron entries and uses the built-in broker.exec service to run a command associated with the entry each time the defined trigger is reached. As with flux-exec(1), these tasks run as direct children of the flux-broker and run outside of the control of any loaded job scheduling service.

  

  Cron
  module
  Cron

  flux-core

• A module that provide a consistent heartbeat (event stream?) that can be subscribed to or used by other things to provide that beat? (please halp!)

  

  heartbeat
  module
  heartbeat

  flux-core

• The job-archive service periodically archives job data in a sqlite database for use by flux-accounting. Parameters may be set in the broker config by the archive table.

  

  Job Archive
  module
  Job Archive

  flux-core

• Prototype job exec service that implements an exec interface for jobs (would be good to understand better what this means).

  

  Job Exec
  module
  Job Exec

  flux-core

• A lookup service to get information about a job from the key value store.

  

  Job Info
  module
  Job Info

  flux-core

• The job-ingest service takes in signed jobspec submitted through flux_job_submit(), and validates the submitting user, the jobspec itself, assigns a job id, commits data to the key value store, and makes a request to submit the job. These actions are done in batches within a batch timeout window, and the job id is returned to the user after the ingest is done.

  

  Job Ingest
  module
  Job Ingest

  flux-core

• The job-list service lists jobs for a local instance (I couldn't find description for this one).

  

  Job List
  module
  Job List

  flux-core

• The job-list service lists jobs for a local instance (I couldn't find description for this one).

  

  Job List
  module
  Job List

  flux-core

• The job-manager service is the high level coordinator of jobs, receiving a notification from the job-injest service of a new job, and returning an id to the user and ensuring the job moves through its lifecycle. The lifecycle includes checking for dependencies, submitting an allocation request to get resources, a start request to the exec service, and interacting with the job exec service to run and get a status result for the job.

  

  Job Manager
  module
  Job Manager

  flux-core

• KVS watch is a service that enables a Flux Event Log to be used for synchronization (more detail here, what does that mean?)

  

  KVS Watch
  module
  KVS Watch

  flux-core

Plugins

• This plugin sets the priority on each job that enters the system based on multiple factors including fair share values. The priority determines the order in which jobs are considered by the scheduler for resource allocation. In addition, the jobtap plugin holds or rejects job requests that exceed user/project specific limits or have exhausted their bank allocations.

  

  Flux accounting multi-factor priority plugin
  plugin
  Flux accounting multi-factor priority plugin

  flux-accounting

• If attributes.system.R exists in jobspec, then bypass scheduler alloc protocol and use R directly (for instance owner use only) (We need a better description here for a layperson)

  

  Allocation bypass plugin
  plugin
  Allocation bypass plugin

  flux-core

• This plugin ensures that resources are never double booked. If they are found to be, a fatal exception is raised on jobs that have resources already granted to others. (Does this ever happen?)

  

  Allocation check plugin
  plugin
  Allocation check plugin

  flux-core

• JobTab

  

  Begin Time plugin
  plugin
  Begin Time plugin

  flux-core

• This plugin ensures we don't start a job until dependency jobs start, complete, or fail (or any generic job with higher priority).

  

  Dependency After plugin
  plugin
  Dependency After plugin

  flux-core

• This plugin tracks jobs in t_submit order per user (I'm not sure I know what this means)

  

  History plugin
  plugin
  History plugin

  flux-core

• This plugin validates job requests against configured duration limits. This means that is uses the job.validate callback to accept or reject job requests.

  

  Limit Duration plugin
  plugin
  Limit Duration plugin

  flux-core

• This plugin validates job requests against configured job size limits.

  

  Limit Job Size plugin
  plugin
  Limit Job Size plugin

  flux-core

• This plugin executes a job manager prolog/epilog for jobs. This usually happens on rank 0 before jobs have been allocated or freed resources.

  

  Perilog plugin
  plugin
  Perilog plugin

  flux-core

• This is the builtin default priority plugin that sets priority to current urgency.

  

  Priority Default plugin
  plugin
  Priority Default plugin

  flux-core

• This plugin holds all submitted jobs.

  

  Submit Hold plugin
  plugin
  Submit Hold plugin

  flux-core

• This plugin ensures that a job's duration doesn't exceed the current resource expiration at the moment of submission.

  

  Validate Duration plugin
  plugin
  Validate Duration plugin

  flux-core

• This plugin likely works with the connector-local module, but I don't understand how.

  

  Connector Local plugin
  plugin
  Connector Local plugin

  flux-core

• This connector plugin is mainly for testing (I don't know the details).

  

  Connector Loop plugin
  plugin
  Connector Loop plugin

  flux-core

• This connector plugin creates a 0MQ inproc socket that communicates with another inproc socket in the same process (normally the flux broker). Pairs of inproc sockets must share a common 0MQ context. This connector uses the libczmq zsock class, which hides creation/sharing of the 0MQ context; therefore, the other inproc socket should be created with zsock also. (When is this used?)

  

  Shared Memory Connector plugin
  plugin
  Shared Memory Connector plugin

  flux-core

• This connector plugin enables connecting to a flux instance via ssh (this is my guess anyway!)

  

  SSH Connector plugin
  plugin
  SSH Connector plugin

  flux-core

• This plugin uses the sched.feasibility RPC to validate a job, allowing jobs that are making infeasible or otherwise invalid requests to be rejected by the scheduler before ingest, instead of when the scheduler attempts to allocate resources for them. This plugin is implemented in Python.

  

  Feasibility Plugin
  plugin
  Feasibility Plugin

  flux-core

• The JobSpec validator plugin uses flux.job.validate_jobspec to validate a submitted jobspec. This plugin is implemented in Python.

  

  JobSpec Validator Plugin
  plugin
  JobSpec Validator Plugin

  flux-core

• This plugin requires that all jobs are new instances of flux, which needs to be done in the case of a batch job or using flux start or flux broker. This plugin is implemented in Python.

  

  Require Instance Validator Plugin
  plugin
  Require Instance Validator Plugin

  flux-core

• This plugin validates jobspec schemas using jsonschema. This plugin is implemented in Python.

  

  Schema Validation Plugin
  plugin
  Schema Validation Plugin

  flux-core

• Apply constraints to incoming jobspec based on broker config. This plugin is implemented in Python.

  

  Frobnicator Constraints Plugin
  plugin
  Frobnicator Constraints Plugin

  flux-core

• This plugin applies defaults to all incoming jobspec based on the broker config.

  

  Frobnicator Defaults Plugin
  plugin
  Frobnicator Defaults Plugin

  flux-core

• This plugin appears to map tasks to cpu (or something like that).

  

  affinity
  plugin
  affinity

  flux-core

• This plugin provides a PMI-1 service so that an MPI or Flux job can bootstrap.

  

  pmi
  plugin
  pmi

  flux-core

• The shell pty plugin parses any pty option, where a pty option is generally a request for a pseudotty, a "device entry that acts like a terminal to the process reading and writing there, but is managed by something else."

  

  pty
  plugin
  pty

  flux-core