azurebatch_helpers Namespace Reference

Classes
class	TimeoutExpiredError

Functions
str	decode_string (Union[str, bytes] string, str encoding="utf-8")
Tuple[str, batchmodels.ImageReference]	select_latest_verified_vm_image_with_node_agent_sku (BatchServiceClient batch_client, str publisher, str offer, str sku_starts_with)
	wait_for_tasks_to_complete (BatchServiceClient batch_client, str job_id, datetime.timedelta timeout)
	print_task_output (BatchServiceClient batch_client, str job_id, List[str] task_ids, str encoding=None)
	print_configuration (ConfigParser config)
str	_read_stream_as_string (stream, str encoding=None)
str	read_task_file_as_string (BatchServiceClient batch_client, str job_id, str task_id, str file_name, str encoding=None)
str	read_compute_node_file_as_string (BatchServiceClient batch_client, str pool_id, str node_id, str file_name, str encoding=None)
	create_pool_if_not_exist (BatchServiceClient batch_client, batchmodels.PoolAddParameter pool)
	create_job (BatchServiceClient batch_service_client, str job_id, str pool_id)
List[batchmodels.ComputeNode]	wait_for_all_nodes_state (BatchServiceClient batch_client, str pool_id, Set[batchmodels.ComputeNodeState] node_state)
str	create_container_and_create_sas (BlobServiceClient blob_service_client, str container_name, ContainerSasPermissions permission, datetime.datetime expiry=None, int timeout=None)
str	create_sas_token (BlobServiceClient blob_service_client, str container_name, str blob_name, BlobSasPermissions permission, datetime.datetime expiry=None, int timeout=None)
str	build_sas_url (BlobServiceClient blob_service_client, str container_name, str blob_name, str sas_token)
str	upload_blob_and_create_sas (BlobServiceClient blob_service_client, str container_name, str blob_name, str file_name, datetime.datetime expiry=None, int timeout=None)
batchmodels.ResourceFile	upload_file_to_container (BlobServiceClient blob_service_client, str container_name, str file_path, int timeout)
	download_blob_from_container (BlobServiceClient blob_service_client, str container_name, str blob_name, str directory_path)
str	generate_unique_resource_name (str resource_prefix)
str	query_yes_no (str question, str default="yes")
	print_batch_exception (batchmodels.BatchErrorException batch_exception)
str	wrap_commands_in_shell (str ostype, List[str] commands)
	wait_for_job_under_job_schedule (BatchServiceClient batch_client, str job_schedule_id, datetime.timedelta timeout)
	wait_for_job_schedule_to_complete (BatchServiceClient batch_client, str job_schedule_id, datetime.datetime timeout_at)

Variables
str	STANDARD_OUT_FILE_NAME = "stdout.txt"
str	STANDARD_ERROR_FILE_NAME = "stderr.txt"
str	SAMPLES_CONFIG_FILE_NAME = "configuration.cfg"

Detailed Description

Batch sample helpers

Function Documentation

_read_stream_as_string()

str azurebatch_helpers._read_stream_as_string ( stream, str encoding = None )

protected

Read stream as string

:param stream: input stream generator
:param encoding: The encoding of the file. The default is utf-8.
:return: The file content.

Here is the caller graph for this function:

build_sas_url()

str azurebatch_helpers.build_sas_url	(	BlobServiceClient	blob_service_client,
		str	container_name,
		str	blob_name,
		str	sas_token )

Builds a signed URL for a blob

:param blob_service_client: The blob service client
:param container_name: The name of the blob container
:param blob_name: The name of the blob
:param sas_token: An SAS token

Here is the caller graph for this function:

create_container_and_create_sas()

str azurebatch_helpers.create_container_and_create_sas	(	BlobServiceClient	blob_service_client,
		str	container_name,
		ContainerSasPermissions	permission,
		datetime.datetime	expiry = None,
		int	timeout = None )

Create a blob sas token

:param blob_service_client: The storage block blob client to use.
:param container_name: The name of the container to upload the blob to.
:param permission The permissions of the SAS token
:param expiry: The SAS expiry time.
:param timeout: timeout in minutes from now for expiry,
    will only be used if expiry is not specified
:return: A SAS token

create_job()

azurebatch_helpers.create_job	(	BatchServiceClient	batch_service_client,
		str	job_id,
		str	pool_id )

Creates a job with the specified ID, associated with the specified pool.

:param batch_service_client: A Batch service client.
:param job_id: The ID for the job.
:param pool_id: The ID for the pool.

Here is the call graph for this function:

create_pool_if_not_exist()

azurebatch_helpers.create_pool_if_not_exist	(	BatchServiceClient	batch_client,
		batchmodels.PoolAddParameter	pool )

Creates the specified pool if it doesn't already exist

:param batch_client: The batch client to use.
:param pool: The pool to create.

create_sas_token()

str azurebatch_helpers.create_sas_token	(	BlobServiceClient	blob_service_client,
		str	container_name,
		str	blob_name,
		BlobSasPermissions	permission,
		datetime.datetime	expiry = None,
		int	timeout = None )

Create a blob sas token

:param blob_service_client: The storage block blob client to use.
:param container_name: The name of the container to upload the blob to.
:param blob_name: The name of the blob to upload the local file to.
:param permission The permissions of the SAS token
:param expiry: The SAS expiry time.
:param timeout: timeout in minutes from now for expiry,
    will only be used if expiry is not specified
:return: A SAS token

Here is the caller graph for this function:

decode_string()

str azurebatch_helpers.decode_string	(	Union[str, bytes]	string,
		str	encoding = "utf-8" )

Decode a string with specified encoding

:param string: string to decode
:param encoding: encoding of string to decode
:return: decoded string

download_blob_from_container()

azurebatch_helpers.download_blob_from_container	(	BlobServiceClient	blob_service_client,
		str	container_name,
		str	blob_name,
		str	directory_path )

Downloads specified blob from the specified Azure Blob storage container.

:param blob_service_client: A blob service client.
:param container_name: The Azure Blob storage container from which to
    download file.
:param blob_name: The name of blob to be downloaded
:param directory_path: The local directory to which to download the file.

generate_unique_resource_name()

str azurebatch_helpers.generate_unique_resource_name

(

str

resource_prefix

)

Generates a unique resource name by appending a time
string after the specified prefix.

:param resource_prefix: The resource prefix to use.
:return: A string with the format "resource_prefix-<time>".

print_batch_exception()

azurebatch_helpers.print_batch_exception

(

batchmodels.BatchErrorException

batch_exception

)

Prints the contents of the specified Batch exception.

:param batch_exception:

Here is the caller graph for this function:

print_configuration()

azurebatch_helpers.print_configuration

(

ConfigParser

config

)

Prints the configuration being used as a dictionary

:param config: The configuration.

print_task_output()

azurebatch_helpers.print_task_output	(	BatchServiceClient	batch_client,
		str	job_id,
		List[str]	task_ids,
		str	encoding = None )

Prints the stdout and stderr for each task specified.

:param batch_client: The batch client to use.
:param job_id: The id of the job to monitor.
:param task_ids: The collection of tasks to print the output for.
:param encoding: The encoding to use when downloading the file.

query_yes_no()

str azurebatch_helpers.query_yes_no	(	str	question,
		str	default = "yes" )

Prompts the user for yes/no input, displaying the specified question text.

:param question: The text of the prompt for input.
:param default: The default if the user hits <ENTER>. Acceptable values
are 'yes', 'no', and None.
:return: 'yes' or 'no'

read_compute_node_file_as_string()

str azurebatch_helpers.read_compute_node_file_as_string	(	BatchServiceClient	batch_client,
		str	pool_id,
		str	node_id,
		str	file_name,
		str	encoding = None )

Reads the specified file as a string.

:param batch_client: The batch client to use.
:param pool_id: The id of the pool.
:param node_id: The id of the node.
:param file_name: The name of the file to read.
:param encoding: The encoding of the file.  The default is utf-8
:return: The file content.

Here is the call graph for this function:

read_task_file_as_string()

str azurebatch_helpers.read_task_file_as_string	(	BatchServiceClient	batch_client,
		str	job_id,
		str	task_id,
		str	file_name,
		str	encoding = None )

Reads the specified file as a string.

:param batch_client: The batch client to use.
:param job_id: The id of the job.
:param task_id: The id of the task.
:param file_name: The name of the file to read.
:param encoding: The encoding of the file. The default is utf-8.
:return: The file content.

Here is the call graph for this function:

select_latest_verified_vm_image_with_node_agent_sku()

Tuple[str, batchmodels.ImageReference] azurebatch_helpers.select_latest_verified_vm_image_with_node_agent_sku	(	BatchServiceClient	batch_client,
		str	publisher,
		str	offer,
		str	sku_starts_with )

Select the latest verified image that Azure Batch supports given
a publisher, offer and sku (starts with filter).

:param batch_client: The batch client to use.
:param publisher: vm image publisher
:param offer: vm image offer
:param sku_starts_with: vm sku starts with filter
:return: (node agent sku id to use, vm image ref to use)

upload_blob_and_create_sas()

str azurebatch_helpers.upload_blob_and_create_sas	(	BlobServiceClient	blob_service_client,
		str	container_name,
		str	blob_name,
		str	file_name,
		datetime.datetime	expiry = None,
		int	timeout = None )

Uploads a file from local disk to Azure Storage and creates
a SAS for it.

:param blob_service_client: The storage block blob client to use.
:param container_name: The name of the container to upload the blob to.
:param blob_name: The name of the blob to upload the local file to.
:param file_name: The name of the local file to upload.
:param expiry: The SAS expiry time.
:param int timeout: timeout in minutes from now for expiry,
    will only be used if expiry is not specified
:return: A SAS URL to the blob with the specified expiry time.

Here is the call graph for this function:

Here is the caller graph for this function:

upload_file_to_container()

batchmodels.ResourceFile azurebatch_helpers.upload_file_to_container	(	BlobServiceClient	blob_service_client,
		str	container_name,
		str	file_path,
		int	timeout )

Uploads a local file to an Azure Blob storage container.

:param blob_service_client: A blob service client.
:param container_name: The name of the Azure Blob storage container.
:param file_path: The local path to the file.
:param timeout: timeout in minutes from now for expiry,
    will only be used if expiry is not specified
:return: A ResourceFile initialized with a SAS URL appropriate for Batch
    tasks.

Here is the call graph for this function:

wait_for_all_nodes_state()

List[batchmodels.ComputeNode] azurebatch_helpers.wait_for_all_nodes_state	(	BatchServiceClient	batch_client,
		str	pool_id,
		Set[batchmodels.ComputeNodeState]	node_state )

Waits for all nodes in pool to reach any specified state in set

:param batch_client: The batch client to use.
:param pool: The pool containing the node.
:param node_state: node states to wait for
:return: list of compute nodes

wait_for_job_schedule_to_complete()

azurebatch_helpers.wait_for_job_schedule_to_complete	(	BatchServiceClient	batch_client,
		str	job_schedule_id,
		datetime.datetime	timeout_at )

Waits for a job schedule to complete.

:param batch_client: The batch client to use.
:param job_schedule_id: The id of the job schedule to monitor.
:param timeout: The maximum amount of time to wait.

wait_for_job_under_job_schedule()

azurebatch_helpers.wait_for_job_under_job_schedule	(	BatchServiceClient	batch_client,
		str	job_schedule_id,
		datetime.timedelta	timeout )

Waits for a job to be created and returns a job id.

:param batch_client: The batch client to use.
:param job_schedule_id: The id of the job schedule to monitor.
:param timeout: The maximum amount of time to wait.

wait_for_tasks_to_complete()

azurebatch_helpers.wait_for_tasks_to_complete	(	BatchServiceClient	batch_client,
		str	job_id,
		datetime.timedelta	timeout )

Waits for all the tasks in a particular job to complete.

:param batch_client: The batch client to use.
:param job_id: The id of the job to monitor.
:param timeout: The maximum amount of time to wait.

wrap_commands_in_shell()

str azurebatch_helpers.wrap_commands_in_shell	(	str	ostype,
		List[str]	commands )

Wrap commands in a shell

:param ostype: OS type, linux or windows
:param commands: list of commands to wrap
:return: a shell wrapping commands