Skip to content

models

Data classes for AWS objects.

AssetRootGroup dataclass

Represents lists of input files, output files and path references grouped under the same root

AssetRootManifest dataclass

Represents asset manifest and a list of output files grouped under the same root

AssetUploadGroup dataclass

Represents all of the information needed to prepare to upload assets

known_asset_paths = field(default_factory=list) class-attribute instance-attribute

List of paths that should not generate warnings

Attachments dataclass

An object that holds the job attachments for a Job

FileStatus

Bases: Enum

Status of local files compared to manifest listed files, comparing hash and time modfied

FileSystemLocation dataclass

DataClass to store File System Location objects

Fleet dataclass

DataClass to store fleet objects

GlobConfig dataclass

Include and Exclude configuration for glob input files

Job dataclass

A non-exhaustive DataClass to store job objects

JobAttachmentS3Settings dataclass

S3-specific Job Attachment settings, configured at the Queue level.

add_root_and_manifest_folder_prefix(path)

Adds “{self.rootPrefix}/{S3_MANIFEST_FOLDER_NAME}/” to the beginning of the path and returns it.

partial_session_action_manifest_prefix(farm_id, queue_id, job_id, step_id, task_id, session_action_id, time) staticmethod

Constructs the partial S3 prefix for storing session action output manifests.

This method creates a hierarchical path structure for organizing output manifests in S3, following the pattern: farm_id/queue_id/job_id/step_id/task_id/timestamp_session_action_id. The timestamp is converted from a float to an ISO datetime string format.

partial_session_action_manifest_prefix_without_task(farm_id, queue_id, job_id, step_id, session_action_id, time) staticmethod

Constructs the partial S3 prefix for storing session action output manifests.

This method creates a hierarchical path structure for organizing output manifests in S3, following the pattern: farm_id/queue_id/job_id/step_id/timestamp_session_action_id. The timestamp is converted from a float to an ISO datetime string format.

ManifestDiff dataclass

Data structure to store new, modified, or deleted files when comparing manifest to a local file system

ManifestDownload dataclass

Data structure to store the S3 and local paths of a manifest

ManifestDownloadResponse dataclass

Data structure to capture the response for manifest download

ManifestMerge dataclass

Data structure to store the S3 and local paths of a manifest

ManifestPathGroup dataclass

Represents paths combined from multiple manifests under the same root path, organized by hash algorithm.

combine_with_group(group)

Adds the content of the given ManifestPathGroup to this ManifestPathGroup

get_all_paths()

Get all paths in this group, regardless of hashing algorithm. Note that this may include duplicates if the same path exists for multiple hashing algorithms.

Returns a sorted list of paths represented as strings.

ManifestProperties dataclass

The assets for a Step under an asset root

as_output_metadata()

Generate S3 metadata for output manifest uploads.

Creates metadata dictionary containing asset root path and optional file system location. Handles non-ASCII characters in paths by JSON-encoding them with ASCII-safe format.

Returns:

Type Description
dict[str, dict[str, str]]

dict[str, str]: S3 metadata dictionary with 'Metadata' key containing: - 'asset-root': ASCII-compatible root path, or JSON-encoded root path for non-ASCII paths - 'asset-root-json': JSON-encoded root path for non-ASCII paths - 'file-system-location-name': Optional file system location name
from_dict(data) classmethod

Create ManifestProperties from a dictionary.

ManifestSnapshot dataclass

Data structure to store the results of a manifest snapshot

OutputFile dataclass

Files for output

PathFormat

Bases: str, Enum

get_host_path_format() classmethod

Get the current path format.

get_host_path_format_string() classmethod

Get a string of the current path format.

PathMappingRule dataclass

destination_path instance-attribute

The path to transform the source path to

source_path instance-attribute

The path we're looking to change

source_path_format instance-attribute

The path format associated with the source path (windows vs posix)

Queue dataclass

DataClass to store queue objects

StorageProfile dataclass

DataClass to store Storage Profile For Queue objects

StorageProfileOperatingSystemFamily

Bases: str, Enum

Case-insensitive enum for the storage profile operating system family type.

get_host_os_family() classmethod

Get the current path format.

UploadManifestInfo dataclass

Structured class for output manifest information.

Attributes:

Name Type Description
output_manifest_path str

The relative path to the uploaded output manifest without root prefix
output_manifest_hash str

The hash of the output manifest content
source_path Optional[str]

Optional source path from the mapping rule