OpenSHMEM

Version and Vendor Query

info_get_version()	Return the major and minor version of the library implementation.
info_get_name()	Return the name string of the library implementation.

shmem4py.shmem.info_get_version()

Return the major and minor version of the library implementation.

Return type:

Tuple[int, int]

shmem4py.shmem.info_get_name()

Return the name string of the library implementation.

Return type:

str

Library Setup and Exit

init()	Allocate and initialize the needed resources.
finalize()	Release all the used resources.
global_exit([status])	Force termination of an entire program.
init_thread([requested])	Initialize the library with support for the provided thread level.
query_thread()	Return the level of thread support provided by the library.
THREAD(value[, names, module, qualname, ...])	Threading support levels.

shmem4py.shmem.init()

Allocate and initialize the needed resources. Collective.

All PEs must call this routine, or init_thread, before any other OpenSHMEM routine. It must be matched with a call to finalize at the end of the program.

Return type:

None

shmem4py.shmem.finalize()

Release all the used resources. Collective.

This only terminates the shmem portion of a program, not the entire program. All processes that represent the PEs will still exist after the call to finalize returns, but they will no longer have access to resources that have been released.

Return type:

None

shmem4py.shmem.global_exit(status=0)

Force termination of an entire program. Can be called by any PE.

Parameters:

status (int) – The exit status of the main program.

Return type:

NoReturn

shmem4py.shmem.init_thread(requested=THREAD_MULTIPLE)

Initialize the library with support for the provided thread level.

Either init or init_thread should be used to initialize the program.

Parameters:

requested (THREAD) – The thread level support requested by the user.

Returns:

The thread level support provided by the implementation.

Return type:

THREAD

shmem4py.shmem.query_thread()

Return the level of thread support provided by the library.

Return type:

THREAD

class shmem4py.shmem.THREAD(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Threading support levels.

SINGLE

A single-threaded program. A hybrid program should not request SINGLE at the initialization call of either OpenSHMEM or MPI but request a different thread level at the initialization call of the other model.

Type:

int

FUNNELED

Allows only the main thread to make communication calls.

Type:

int

SERIALIZED

Communication calls are not made concurrently by multiple threads.

Type:

int

MULTIPLE

The program may be multithreaded and any thread may invoke the OpenSHMEM interfaces.

Type:

int

Accessibility Queries

my_pe()	Return the number of the calling PE.
n_pes()	Return the number of PEs running in a program.
pe_accessible(pe)	Return whether a PE is accessible.
addr_accessible(addr, pe)	Return whether a local array is accessible from the specified remote PE.
ptr(target, pe)	Return a local view to a symmetric array on the specified PE.

shmem4py.shmem.my_pe()

Return the number of the calling PE.

Return type:

int

shmem4py.shmem.n_pes()

Return the number of PEs running in a program.

Return type:

int

shmem4py.shmem.pe_accessible(pe)

Return whether a PE is accessible.

Parameters:

pe (int) – The PE number to check for accessibility from the local PE.

Return type:

bool

shmem4py.shmem.addr_accessible(addr, pe)

Return whether a local array is accessible from the specified remote PE.

Parameters:

• addr (NDArray[Any]) – Local array object to query.

• pe (int) – The id of a remote PE.

Return type:

bool

shmem4py.shmem.ptr(target, pe)

Return a local view to a symmetric array on the specified PE.

Parameters:

• target (NDArray[T]) – The symmetric destination array.

• pe (int) – The PE number on which target is to be accessed.

Returns:

A local pointer to the remotely accessible target array is returned when it can be accessed using memory loads and stores. Otherwise, None is returned.

Return type:

NDArray[T] | None

Memory Management

alloc(count[, size, align, hints, clear])	Return memory allocated from the symmetric heap.
free(mem)	Deallocate memory of mem.
fromalloc(mem[, shape, dtype, order])	Return a NumPy array interpreted from the buffer allocated in the symmetric memory.
new_array(shape[, dtype, order, align, ...])	Return a new NumPy array allocated in the symmetric memory.
del_array(a)	Delete the array.
array(obj[, dtype, order, align, hints])	Return a new NumPy array allocated in the symmetric memory and initialize contents with obj.
empty(shape[, dtype, order, align, hints])	Return a new empty NumPy array allocated in the symmetric memory.
zeros(shape[, dtype, order, align, hints])	Return a new 0-initialized NumPy array allocated in the symmetric memory.
ones(shape[, dtype, order, align, hints])	Return a new 1-initialized NumPy array allocated in the symmetric memory.
full(shape, fill_value[, dtype, order, ...])	Return a new fill_value-initialized NumPy array allocated in the symmetric memory.
MALLOC(value[, names, module, qualname, ...])	Memory allocation hints.

shmem4py.shmem.alloc(count, size=1, align=None, hints=None, clear=True)

Return memory allocated from the symmetric heap.

Parameters:

• count (int) – Number of elements to allocate.

• size (int) – Size of each element in bytes.

• align (int | None) – Byte alignment of the block allocated from the symmetric heap.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator.

• clear (bool) – If True, the allocated memory is cleared to zero.

Return type:

memoryview

shmem4py.shmem.free(mem)

Deallocate memory of mem.

Parameters:

mem (memoryview | NDArray[Any]) – The object to be deallocated.

Return type:

None

shmem4py.shmem.fromalloc(mem, shape=None, dtype=None, order='C')

Return a NumPy array interpreted from the buffer allocated in the symmetric memory.

Parameters:

• mem (memoryview) – The memory to be interpreted as a NumPy array.

• shape (int | Sequence[int] | None) – The shape of the array. If None, the shape is inferred from the size of the memory.

• dtype (DTypeLike) – The data type of the array. If None, the data type is inferred from the memory contents.

• order (Literal['C', 'F']) – The memory layout of the array. If 'C', the array is contiguous in memory (row major). If 'F', the array is Fortran contiguous (column major).

Return type:

NDArray[Any]

shmem4py.shmem.new_array(shape, dtype=float, order='C', *, align=None, hints=None, clear=True)

Return a new NumPy array allocated in the symmetric memory.

Parameters:

• shape (int | Sequence[int]) – The shape of the array.

• dtype (DTypeLike) – The data type of the array.

• order (Literal['C', 'F']) – The memory layout of the array. If 'C', the array is contiguous in memory (row major). If 'F', the array is Fortran contiguous (column major).

• align (int | None) – Byte alignment of the block allocated in the symmetric memory. Keyword argument only.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator. Keyword argument only.

• clear (bool) – If True, the allocated memory is cleared to zero. Keyword argument only.

Return type:

NDArray[Any]

shmem4py.shmem.del_array(a)

Delete the array.

Parameters:

a (NDArray[Any]) – The array to be deleted.

Return type:

None

shmem4py.shmem.array(obj, dtype=None, *, order='K', align=None, hints=None)

Return a new NumPy array allocated in the symmetric memory and initialize contents with obj.

Parameters:

• obj (Any) – The object from which a NumPy array is to be initialized.

• dtype (DTypeLike) – The data type of the array. If None, the data type is inferred from the memory contents.

• order (Literal['K', 'A', 'C', 'F']) – The memory layout of the array. See numpy.array for the explanation of the options. Keyword argument only.

• align (int | None) – Byte alignment of the block allocated in the symmetric memory. Keyword argument only.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator. Keyword argument only.

Return type:

NDArray[Any]

shmem4py.shmem.empty(shape, dtype=float, order='C', *, align=None, hints=None)

Return a new empty NumPy array allocated in the symmetric memory.

Parameters:

• shape (int | Sequence[int]) – The shape of the array.

• dtype (DTypeLike) – The data type of the array.

• order (Literal['C', 'F']) – The memory layout of the array. If 'C', the array is contiguous in memory (row major). If 'F', the array is Fortran contiguous (column major).

• align (int | None) – Byte alignment of the block allocated in the symmetric memory. Keyword argument only.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator. Keyword argument only.

Return type:

NDArray[Any]

shmem4py.shmem.zeros(shape, dtype=float, order='C', *, align=None, hints=None)

Return a new 0-initialized NumPy array allocated in the symmetric memory.

Parameters:

• shape (int | Sequence[int]) – The shape of the array.

• dtype (DTypeLike) – The data type of the array.

• order (Literal['C', 'F']) – The memory layout of the array. If 'C', the array is contiguous in memory (row major). If 'F', the array is Fortran contiguous (column major).

• align (int | None) – Byte alignment of the block allocated in the symmetric memory. Keyword argument only.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator. Keyword argument only.

Return type:

NDArray[Any]

shmem4py.shmem.ones(shape, dtype=float, order='C', *, align=None, hints=None)

Return a new 1-initialized NumPy array allocated in the symmetric memory.

Parameters:

• shape (int | Sequence[int]) – The shape of the array.

• dtype (DTypeLike) – The data type of the array.

• order (Literal['C', 'F']) – The memory layout of the array. If 'C', the array is contiguous in memory (row major). If 'F', the array is Fortran contiguous (column major).

• align (int | None) – Byte alignment of the block allocated in the symmetric memory. Keyword argument only.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator. Keyword argument only.

Return type:

NDArray[Any]

shmem4py.shmem.full(shape, fill_value, dtype=None, order='C', *, align=None, hints=None)

Return a new fill_value-initialized NumPy array allocated in the symmetric memory.

Parameters:

• shape (int | Sequence[int]) – The shape of the array.

• fill_value (int | float | complex | number) – The value to fill the array with.

• dtype (DTypeLike) – The data type of the array.

• order (Literal['C', 'F']) – The memory layout of the array. If 'C', the array is contiguous in memory (row major). If 'F', the array is Fortran contiguous (column major).

• align (int | None) – Byte alignment of the block allocated in the symmetric memory. Keyword argument only.

• hints (int | None) – A bit array of hints provided by the user to the implementation. Valid hints are defined as enumerations in MALLOC and can be combined using the bitwise OR operator. Keyword argument only.

Return type:

NDArray[Any]

class shmem4py.shmem.MALLOC(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Memory allocation hints.

ATOMICS_REMOTE

The allocated memory will be used for atomic variables.

Type:

int

SIGNAL_REMOTE

The allocated memory will be used for signal variables.

Type:

int

Team Management

Team([team])	Team management.
Team.destroy()	Destroy the team.
Team.split_strided([start, stride, size, config])	Return a new team from a subset of the existing parent team PEs.
Team.get_config()	Return the configuration parameters of the team.
Team.my_pe()	Return the number of the calling PE within the team.
Team.n_pes()	Return the number of PEs in the team.
Team.translate_pe([pe, team])	Translate a given PE number from one team to the corresponding PE number in another team.
Team.create_ctx([options])	Create a communication context from the team.
Team.sync()	Register the arrival of a PE at a synchronization point.

class shmem4py.shmem.Team(team=None)

Team management.

Parameters:

team (Optional[Union[Team, TeamHandle]])

Return type:

Team

destroy()

Destroy the team.

Return type:

None

split_strided(start=0, stride=1, size=None, config=None, **kwargs)

Return a new team from a subset of the existing parent team PEs.

This routine must be called by all PEs in the parent team.

Parameters:

• start (int) – The lowest PE number of the subset of PEs from the parent team that will form the new team.

• stride (int) – The stride between team PE numbers in the parent team that comprise the subset of PEs that will form the new team.

• size (int | None) – The number of PEs from the parent team in the subset of PEs that will form the new team. If None, the size is automatically determined.

• config (Mapping[str, int] | None) – Configuration parameters for the new team. Currently, only SHMEM_TEAM_NUM_CONTEXTS key is supported.

• **kwargs (int) – Additional configuration parameters for the new team.

Return type:

Team

get_config()

Return the configuration parameters of the team.

Return type:

Dict[str, int]

my_pe()

Return the number of the calling PE within the team.

Return type:

int

n_pes()

Return the number of PEs in the team.

Return type:

int

translate_pe(pe=None, team=None)

Translate a given PE number from one team to the corresponding PE number in another team.

Parameters:

• pe (int | None) – PE number in the source team. If None, defaults to the calling PE number.

• team (Team | None) – Destination team. If None, defaults to the world team.

Return type:

int

create_ctx(options=0)

Create a communication context from the team.

Parameters:

options (int) – The set of options requested for the given context. Valid options are the enumerations listed in the CTX class. Multiple options may be requested by combining them with a bitwise OR operation. 0 can be used if no options are requested.

Return type:

Ctx

sync()

Register the arrival of a PE at a synchronization point.

This routine does not return until all other PEs in a given team or active set arrive at this synchronization point.

Return type:

None

Communication Management

Ctx([ctx])	Communication context.
Ctx.create([options, team])	Return a new communication context.
Ctx.destroy()	Destroy the communication context.
Ctx.get_team()	Retrieve the team associated with the communication context.
Ctx.fence()	Ensure ordering of delivery of operations on symmetric data objects.
Ctx.quiet()	Wait for completion of outstanding operations on symmetric data objects issued by a PE.
CTX(value[, names, module, qualname, type, ...])	Context creation options.

class shmem4py.shmem.Ctx(ctx=None)

Communication context.

Parameters:

ctx (Optional[Union[Ctx, CtxHandle]])

Return type:

Ctx

static create(options=0, team=None)

Return a new communication context.

Parameters:

• options (int) – The set of options requested for the given context. Valid options are the enumerations listed in the CTX class. Multiple options may be requested by combining them with a bitwise OR operation. 0 can be used if no options are requested.

• team (Team | None) – If the team is specified, the communication context is created from this team.

Return type:

Ctx

destroy()

Destroy the communication context.

Return type:

None

get_team()

Retrieve the team associated with the communication context.

Return type:

Team

fence()

Ensure ordering of delivery of operations on symmetric data objects.

All operations on symmetric data objects issued to a particular PE on the given context prior to the call to fence are guaranteed to be delivered before any subsequent operations on symmetric data objects to the same PE.

Return type:

None

quiet()

Wait for completion of outstanding operations on symmetric data objects issued by a PE.

Ensures completion of all operations on symmetric data objects issued by the calling PE on the given context.

Return type:

None

class shmem4py.shmem.CTX(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Context creation options.

PRIVATE

The given context will be used only by the thread that created it.

Type:

int

SERIALIZED

The given context is shareable but will not be used by multiple threads concurrently.

Type:

int

NOSTORE

quiet and fence operations performed on the given context are not required to enforce completion and ordering of memory store operations.

Type:

int

Remote Memory Access

put(target, source, pe[, size, ctx])	Copy data from local source to target on PE pe.
get(target, source, pe[, size, ctx])	Copy data from source on PE pe to local target.
iput(target, source, pe[, tst, sst, size, ctx])	Copy strided data from local source to target on PE pe.
iget(target, source, pe[, tst, sst, size, ctx])	Copy strided data from source on PE pe to local target.
put_nbi(target, source, pe[, size, ctx])	Nonblocking copy data from local source to target on PE pe.
get_nbi(target, source, pe[, size, ctx])	Nonblocking copy data from source on PE pe to local target.

shmem4py.shmem.put(target, source, pe, size=None, ctx=None)

Copy data from local source to target on PE pe.

Parameters:

• target (NDArray[T]) – Symmetric destination array.

• source (NDArray[T]) – Local array containing the data to be copied.

• pe (int) – PE number of the remote PE.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – A context handle specifying the context on which to perform the operation.

Return type:

None

shmem4py.shmem.get(target, source, pe, size=None, ctx=None)

Copy data from source on PE pe to local target.

Parameters:

• target (NDArray[T]) – Local array to be updated.

• source (NDArray[T]) – Symmetric source array.

• pe (int) – PE number of the remote PE.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – A context handle specifying the context on which to perform the operation.

Return type:

None

shmem4py.shmem.iput(target, source, pe, tst=1, sst=1, size=None, ctx=None)

Copy strided data from local source to target on PE pe.

Parameters:

• target (NDArray[T]) – Symmetric destination array.

• source (NDArray[T]) – Local array containing the data to be copied.

• pe (int) – PE number of the remote PE.

• tst (int) – The stride between consecutive elements of the target array. The stride is scaled by the element size of the target array. A value of 1 indicates contiguous data.

• sst (int) – The stride between consecutive elements of the source array. The stride is scaled by the element size of the source array. A value of 1 indicates contiguous data.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – A context handle specifying the context on which to perform the operation.

Return type:

None

shmem4py.shmem.iget(target, source, pe, tst=1, sst=1, size=None, ctx=None)

Copy strided data from source on PE pe to local target.

Parameters:

• target (NDArray[T]) – Local array to be updated.

• source (NDArray[T]) – Symmetric source array.

• pe (int) – PE number of the remote PE.

• tst (int) – The stride between consecutive elements of the target array. The stride is scaled by the element size of the target array. A value of 1 indicates contiguous data.

• sst (int) – The stride between consecutive elements of the source array. The stride is scaled by the element size of the source array. A value of 1 indicates contiguous data.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – A context handle specifying the context on which to perform the operation.

Return type:

None

shmem4py.shmem.put_nbi(target, source, pe, size=None, ctx=None)

Nonblocking copy data from local source to target on PE pe.

Parameters:

• target (NDArray[T]) – Symmetric destination array.

• source (NDArray[T]) – Local array containing the data to be copied.

• pe (int) – PE number of the remote PE.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – A context handle specifying the context on which to perform the operation.

Return type:

None

shmem4py.shmem.get_nbi(target, source, pe, size=None, ctx=None)

Nonblocking copy data from source on PE pe to local target.

Parameters:

• target (NDArray[T]) – Local array to be updated.

• source (NDArray[T]) – Symmetric source array.

• pe (int) – PE number of the remote PE.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – A context handle specifying the context on which to perform the operation.

Return type:

None

Atomic Memory Operations

atomic_op(target, value, op, pe[, ctx])	Perform operation op on value and target on PE pe.
atomic_fetch_op(target, value, op, pe[, ctx])	Perform operation op on value and target on PE pe and return target's prior value.
atomic_fetch_op_nbi(fetch, target, value, op, pe)	Perform operation op on value and target on PE pe and fetch target's prior value to fetch.
AMO(value[, names, module, qualname, type, ...])	Atomic Memory Operations.

atomic_set(target, value, pe[, ctx])	Write value into target on PE pe.
atomic_inc(target, pe[, ctx])	Increment target array element on PE pe.
atomic_add(target, value, pe[, ctx])	Add value to target on PE pe and atomically update target.
atomic_and(target, value, pe[, ctx])	Perform bitwise AND on value and target on PE pe.
atomic_or(target, value, pe[, ctx])	Perform bitwise OR on value and target on PE pe.
atomic_xor(target, value, pe[, ctx])	Perform bitwise XOR on value and target on PE pe.
atomic_fetch(source, pe[, ctx])	Return the value of a source on PE pe.
atomic_swap(target, value, pe[, ctx])	Write value into target on PE pe and return the prior value.
atomic_compare_swap(target, cond, value, pe)	Conditionally update target on PE pe and return its prior value.
atomic_fetch_inc(target, pe[, ctx])	Increment target on PE pe and return its prior value.
atomic_fetch_add(target, value, pe[, ctx])	Add value to target on PE pe and return its prior value.
atomic_fetch_and(target, value, pe[, ctx])	Perform a bitwise AND on value and target at PE pe and return target's prior value.
atomic_fetch_or(target, value, pe[, ctx])	Perform a bitwise OR on value and target at PE pe and return target's prior value.
atomic_fetch_xor(target, value, pe[, ctx])	Perform a bitwise XOR on value and target at PE pe and return target's prior value.
atomic_fetch_nbi(fetch, source, pe[, ctx])	Fetch the value of source on PE pe to local fetch.
atomic_swap_nbi(fetch, target, value, pe[, ctx])	Write value into target on PE pe and fetch its prior value to fetch.
atomic_compare_swap_nbi(fetch, target, cond, ...)	Conditionally update target and fetch its prior value to fetch.
atomic_fetch_inc_nbi(fetch, target, pe[, ctx])	Increment target on PE pe and fetch its prior value to fetch.
atomic_fetch_add_nbi(fetch, target, value, pe)	Add value to target on PE pe and fetch its prior value to fetch.
atomic_fetch_and_nbi(fetch, target, value, pe)	Perform bitwise AND on target on PE pe and fetch its prior value to fetch.
atomic_fetch_or_nbi(fetch, target, value, pe)	Perform bitwise OR on target on PE pe and fetch its prior value to fetch.
atomic_fetch_xor_nbi(fetch, target, value, pe)	Perform bitwise XOR on target on PE pe and fetch its prior value to fetch.

shmem4py.shmem.atomic_op(target, value, op, pe, ctx=None)

Perform operation op on value and target on PE pe.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the operation.

• op (AMO) – The operation to be performed.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch_op(target, value, op, pe, ctx=None)

Perform operation op on value and target on PE pe and return target’s prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the operation.

• op (AMO) – The operation to be performed.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_op_nbi(fetch, target, value, op, pe, ctx=None)

Perform operation op on value and target on PE pe and fetch target’s prior value to fetch.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the operation.

• op (AMO) – The operation to be performed.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

class shmem4py.shmem.AMO(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Atomic Memory Operations.

SET

Set.

Type:

str

INC

Increment.

Type:

str

ADD

Add.

Type:

str

AND

Bitwise AND.

Type:

str

OR

Bitwise OR.

Type:

str

XOR

Bitwise XOR.

Type:

str

shmem4py.shmem.atomic_set(target, value, pe, ctx=None)

Write value into target on PE pe.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 where data will be written.

• value (int | float | complex | number) – The operand to the atomic set operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_inc(target, pe, ctx=None)

Increment target array element on PE pe.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the element that will be modified.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_add(target, value, pe, ctx=None)

Add value to target on PE pe and atomically update target.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the element that will be modified.

• value (int | float | complex | number) – The operand to the atomic add operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_and(target, value, pe, ctx=None)

Perform bitwise AND on value and target on PE pe.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the element that will be modified.

• value (int | float | complex | number) – The operand to the bitwise AND operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_or(target, value, pe, ctx=None)

Perform bitwise OR on value and target on PE pe.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the element that will be modified.

• value (int | float | complex | number) – The operand to the bitwise OR operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_xor(target, value, pe, ctx=None)

Perform bitwise XOR on value and target on PE pe.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the element that will be modified.

• value (int | float | complex | number) – The operand to the bitwise XOR operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch(source, pe, ctx=None)

Return the value of a source on PE pe.

Parameters:

• source – Symmetric array of size 1 containing the element that will be fetched.

• pe (int) – The PE number from which source is to be fetched.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_swap(target, value, pe, ctx=None)

Write value into target on PE pe and return the prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The value to be atomically written to the remote PE.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_compare_swap(target, cond, value, pe, ctx=None)

Conditionally update target on PE pe and return its prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• cond (int | float | complex | number) – cond is compared to the remote target value. If cond and the remote target are equal, then value is swapped into the target; otherwise, the target is unchanged.

• value (int | float | complex | number) – The value to be atomically written to the remote PE.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_inc(target, pe, ctx=None)

Increment target on PE pe and return its prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_add(target, value, pe, ctx=None)

Add value to target on PE pe and return its prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the atomic fetch-and-add operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_and(target, value, pe, ctx=None)

Perform a bitwise AND on value and target at PE pe and return target’s prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the bitwise AND operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_or(target, value, pe, ctx=None)

Perform a bitwise OR on value and target at PE pe and return target’s prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the bitwise OR operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_xor(target, value, pe, ctx=None)

Perform a bitwise XOR on value and target at PE pe and return target’s prior value.

Parameters:

• target (NDArray[Any]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the bitwise XOR operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

int | float | complex | number

shmem4py.shmem.atomic_fetch_nbi(fetch, source, pe, ctx=None)

Fetch the value of source on PE pe to local fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• source (NDArray[T]) – Symmetric array of size 1 containing the element that will be fetched.

• pe (int) – The PE number from which source is to be fetched.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_swap_nbi(fetch, target, value, pe, ctx=None)

Write value into target on PE pe and fetch its prior value to fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The value to be atomically written to the remote PE.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_compare_swap_nbi(fetch, target, cond, value, pe, ctx=None)

Conditionally update target and fetch its prior value to fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• cond (int | float | complex | number) – cond is compared to the remote target value. If cond and the remote target are equal, then value is swapped into the target; otherwise, the target is unchanged.

• value (int | float | complex | number) – The value to be atomically written to the remote PE.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch_inc_nbi(fetch, target, pe, ctx=None)

Increment target on PE pe and fetch its prior value to fetch.

Nonblocking.

The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch_add_nbi(fetch, target, value, pe, ctx=None)

Add value to target on PE pe and fetch its prior value to fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The value to be the atomic fetch-and-add operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch_and_nbi(fetch, target, value, pe, ctx=None)

Perform bitwise AND on target on PE pe and fetch its prior value to fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the bitwise AND operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch_or_nbi(fetch, target, value, pe, ctx=None)

Perform bitwise OR on target on PE pe and fetch its prior value to fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the bitwise OR operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.atomic_fetch_xor_nbi(fetch, target, value, pe, ctx=None)

Perform bitwise XOR on target on PE pe and fetch its prior value to fetch.

Nonblocking. The operation is considered complete after a subsequent call to quiet.

Parameters:

• fetch (NDArray[T]) – Local array of size 1 to be updated.

• target (NDArray[T]) – Symmetric array of size 1 containing the destination value.

• value (int | float | complex | number) – The operand to the bitwise XOR operation.

• pe (int) – The PE number on which target is to be updated.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

Signaling Operations

new_signal()	Create a signal data object.
del_signal(signal)	Delete a signal data object.
signal_fetch(signal)	Fetch the signal update on a local data object.
put_signal(target, source, pe, signal, ...)	Copy local source to target on PE pe and update a remote flag to signal completion.
put_signal_nbi(target, source, pe, signal, ...)	Copy local source to target on PE pe and update a remote flag to signal completion.
SIGNAL(value[, names, module, qualname, ...])	Signal operations.

shmem4py.shmem.new_signal()

Create a signal data object.

Return type:

SigAddr

shmem4py.shmem.del_signal(signal)

Delete a signal data object.

Parameters:

signal (SigAddr) – A signal data object to be deleted.

Return type:

None

shmem4py.shmem.signal_fetch(signal)

Fetch the signal update on a local data object.

Parameters:

signal (SigAddr) – Local, remotely accessible signal variable.

Returns:

The contents of the signal data object at the calling PE.

Return type:

int

shmem4py.shmem.put_signal(target, source, pe, signal, value, sigop, size=None, ctx=None)

Copy local source to target on PE pe and update a remote flag to signal completion.

Parameters:

• target (NDArray[T]) – The symmetric destination array to be updated on the remote PE.

• source (NDArray[T]) – Local array containing the data to be copied.

• pe (int) – PE number of the remote PE.

• signal (SigAddr) – Symmetric signal object to be updated on the remote PE as a signal.

• value (int) – The value that is used for updating the remote signal data object.

• sigop (SIGNAL) – Signal operator that represents the type of update to be performed on the remote signal data object.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

shmem4py.shmem.put_signal_nbi(target, source, pe, signal, value, sigop, size=None, ctx=None)

Copy local source to target on PE pe and update a remote flag to signal completion. Nonblocking.

This routine returns after initiating the operation. The operation is considered complete after a subsequent call to quiet.

Parameters:

• target (NDArray[T]) – The symmetric destination array to be updated on the remote PE.

• source (NDArray[T]) – Local array containing the data to be copied.

• pe (int) – PE number of the remote PE.

• signal (SigAddr) – Symmetric signal object to be updated on the remote PE as a signal.

• value (int) – The value that is used for updating the remote signal data object.

• sigop (SIGNAL) – Signal operator that represents the type of update to be performed on the remote signal data object.

• size (int | None) – Number of elements to copy.

• ctx (Ctx | None) – The context on which to perform the operation. If None, the default context is used.

Return type:

None

class shmem4py.shmem.SIGNAL(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Signal operations.

SET

An update to signal data object is an atomic set operation. It writes an unsigned 64-bit value as a signal into the signal data object on a remote PE as an atomic operation.

Type:

int

ADD

An update to signal data object is an atomic add operation. It adds an unsigned 64-bit value as a signal into the signal data object on a remote PE as an atomic operation.

Type:

int

Collective Operations

barrier_all()	Register the arrival of a PE at a barrier, complete updates, wait for others.
sync_all()	Register the arrival of a PE at a synchronization point, wait for all others.
sync([team])	Register the arrival of a PE at a synchronization point, wait for others.
broadcast(target, source, root[, size, team])	Copy the source from root to target on participating PEs.
collect(target, source[, size, team])	Concatenate blocks of data from multiple PEs to an array in every PE participating in the collective routine.
fcollect(target, source[, size, team])	Concatenate blocks of data from multiple PEs to an array in every PE participating in the collective routine.
alltoall(target, source[, size, team])	Exchange data elements with all other participating PEs.
alltoalls(target, source[, tst, sst, size, team])	Exchange strided data elements with all other participating PEs.
reduce(target, source[, op, size, team])	Perform a specified reduction across a set of PEs.
OP(value[, names, module, qualname, type, ...])	Reduction operation.
and_reduce(target, source[, size, team])	Perform a bitwise AND reduction across a set of PEs.
or_reduce(target, source[, size, team])	Perform a bitwise OR reduction across a set of PEs.
xor_reduce(target, source[, size, team])	Perform a bitwise exclusive OR (XOR) reduction across a set of PEs.
max_reduce(target, source[, size, team])	Perform a maximum-value reduction across a set of PEs.
min_reduce(target, source[, size, team])	Perform a minimum-value reduction across a set of PEs.
sum_reduce(target, source[, size, team])	Perform a sum reduction across a set of PEs.
prod_reduce(target, source[, size, team])	Perform a product reduction across a set of PEs.

shmem4py.shmem.barrier_all()

Register the arrival of a PE at a barrier, complete updates, wait for others.

This routine blocks the calling PE until all PEs have called barrier_all. Prior to synchronizing with other PEs, barrier_all ensures completion of all previously issued memory stores and remote memory updates issued on the default context.

Return type:

None

shmem4py.shmem.sync_all()

Register the arrival of a PE at a synchronization point, wait for all others.

This routine blocks the calling PE until all PEs in the world team have called sync_all.

Return type:

None

shmem4py.shmem.sync(team=None)

Register the arrival of a PE at a synchronization point, wait for others.

This routine does not return until all other PEs in a given team or active set arrive at this synchronization point.

Parameters:

team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.broadcast(target, source, root, size=None, team=None)

Copy the source from root to target on participating PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array.

• source (NDArray[T]) – Symmetric source array.

• root (int) – PE number within the team or active set from which the data is copied.

• size (int | None) – The number of elements to be copied.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.collect(target, source, size=None, team=None)

Concatenate blocks of data from multiple PEs to an array in every PE participating in the collective routine.

size can vary from PE to PE; MPI_Allgatherv equivalent.

Performs a collective operation to concatenate size data items from the source array into the target array.

Parameters:

• target (NDArray[T]) – Symmetric destination array large enough to accept the concatenation of the source arrays on all participating PEs.

• source (NDArray[T]) – Symmetric source array.

• size (int | None) – The number of elements to be communicated.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.fcollect(target, source, size=None, team=None)

Concatenate blocks of data from multiple PEs to an array in every PE participating in the collective routine.

size must be the same value in all participating PEs; MPI_Allgather equivalent.

Parameters:

• target (NDArray[T]) – Symmetric destination array large enough to accept the concatenation of the source arrays on all participating PEs.

• source (NDArray[T]) – Symmetric source array.

• size (int | None) – The number of elements to be communicated.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.alltoall(target, source, size=None, team=None)

Exchange data elements with all other participating PEs.

The total size of each PE’s source object and target object is size times the size of an element times N, where N equals the number of PEs participating in the operation. The source object contains N blocks of data (where the size of each block is defined by size) and each block of data is sent to a different PE.

Parameters:

• target – Symmetric destination array large enough to receive the combined total of size elements from each PE in the active set.

• source – Symmetric source array that contains size elements of data for each PE in the active set, ordered according to destination PE.

• size – The number of elements to exchange for each PE.

• team – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.alltoalls(target, source, tst=1, sst=1, size=None, team=None)

Exchange strided data elements with all other participating PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array large enough to receive the combined total of size elements from each PE in the active set.

• source (NDArray[T]) – Symmetric source array that contains size elements of data for each PE in the active set, ordered according to destination PE.

• tst (int) – The stride between consecutive elements of the target data object. The stride is scaled by the element size.

• sst (int) – The stride between consecutive elements of the source data object. The stride is scaled by the element size.

• size (int | None) – The number of elements to exchange for each PE.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.reduce(target, source, op=OP_SUM, size=None, team=None)

Perform a specified reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• op (OP) – The reduction operation to perform.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

class shmem4py.shmem.OP(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Reduction operation.

AND

Bitwise AND.

Type:

str

OR

Bitwise OR.

Type:

str

XOR

Bitwise XOR.

Type:

str

MAX

Maximum value.

Type:

str

MIN

Minimum value.

Type:

str

SUM

Sum.

Type:

str

PROD

Product.

Type:

str

shmem4py.shmem.and_reduce(target, source, size=None, team=None)

Perform a bitwise AND reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.or_reduce(target, source, size=None, team=None)

Perform a bitwise OR reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.xor_reduce(target, source, size=None, team=None)

Perform a bitwise exclusive OR (XOR) reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.max_reduce(target, source, size=None, team=None)

Perform a maximum-value reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.min_reduce(target, source, size=None, team=None)

Perform a minimum-value reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.sum_reduce(target, source, size=None, team=None)

Perform a sum reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

shmem4py.shmem.prod_reduce(target, source, size=None, team=None)

Perform a product reduction across a set of PEs.

Parameters:

• target (NDArray[T]) – Symmetric destination array of length at least size elements, where the result of the reduction routine will be stored.

• source (NDArray[T]) – Symmetric source array of length at least size elements, that contains one element for each separate reduction routine.

• size (int | None) – The number of elements to perform the reduction on.

• team (Team | None) – The team over which to perform the operation.

Return type:

None

Point-To-Point Synchronization

wait_until(ivar, cmp, value)	Wait until a variable satisfies a condition.
wait_until_all(ivars, cmp, value[, status])	Wait until all variables satisfy a condition.
wait_until_any(ivars, cmp, value[, status])	Wait until any one variable satisfies a condition.
wait_until_some(ivars, cmp, value[, status])	Wait until at least one variable satisfies a condition.
wait_until_all_vector(ivars, cmp, values[, ...])	Wait until all variables satisfy the specified conditions.
wait_until_any_vector(ivars, cmp, values[, ...])	Wait until any one variable satisfies the specified conditions.
wait_until_some_vector(ivars, cmp, values[, ...])	Wait until at least one variable satisfies the specified conditions.
test(ivar, cmp, value)	Indicate whether a variable on the local PE meets a condition.
test_all(ivars, cmp, value[, status])	Indicate whether all variables on the local PE meet a condition.
test_any(ivars, cmp, value[, status])	Indicate whether any one variable on the local PE meets a condition.
test_some(ivars, cmp, value[, status])	Indicate whether at least one variable on the local PE meets a condition.
test_all_vector(ivars, cmp, values[, status])	Indicate whether all variables on the local PE meet the specified conditions.
test_any_vector(ivars, cmp, values[, status])	Indicate whether any one variable on the local PE meets its specified condition.
test_some_vector(ivars, cmp, values[, status])	Indicate whether at least one variable on the local PE meets its specified condition.
signal_wait_until(signal, cmp, value)	Wait for a variable on the local PE to change from a signaling operation.
CMP(value[, names, module, qualname, type, ...])	Comparison operator.

shmem4py.shmem.wait_until(ivar, cmp, value)

Wait until a variable satisfies a condition.

Blocks until the value ivar satisfies the condition ivar cmp value at the calling PE, where cmp is the comparison operator.

Parameters:

• ivar (NDArray[Any]) – Symmetric array of size 1 containing the element that will be compared.

• cmp (CMP) – The comparison operator that compares ivar with value.

• value (int | float | complex | number) – The value to be compared with ivar.

Return type:

None

shmem4py.shmem.wait_until_all(ivars, cmp, value, status=None)

Wait until all variables satisfy a condition.

Blocks until all values specified in ivars not excluded by status satisfy the condition ivars[i] cmp value at the calling PE, where cmp is the comparison operator.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be compared.

• cmp (CMP) – The comparison operator that compares elements of ivars with value.

• value (int | float | complex | number) – The value to be compared with elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the wait set. Nonzero values exclude the corresponding element from the wait set.

Return type:

None

shmem4py.shmem.wait_until_any(ivars, cmp, value, status=None)

Wait until any one variable satisfies a condition.

Blocks until any one entry in the wait set specified by ivars not excluded by status satisfies the condition ivars[i] cmp value at the calling PE, where cmp is the comparison operator.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be compared.

• cmp (CMP) – The comparison operator that compares elements of ivars with value.

• value (int | float | complex | number) – The value to be compared with elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the wait set. Nonzero values exclude the corresponding element from the wait set.

Returns:

The index of entry i of ivars that satisfies the condition.

Return type:

int | None

shmem4py.shmem.wait_until_some(ivars, cmp, value, status=None)

Wait until at least one variable satisfies a condition.

Blocks until at least one entry in the wait set specified by ivars not excluded by status satisfies the condition ivars[i] cmp value at the calling PE, where cmp is the comparison operator.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be compared.

• cmp (CMP) – The comparison operator that compares elements of ivars with value.

• value (int | float | complex | number) – The value to be compared with elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the wait set. Nonzero values exclude the corresponding element from the wait set.

Returns:

Indices of entries of ivars that satisfy the condition.

Return type:

List[int]

shmem4py.shmem.wait_until_all_vector(ivars, cmp, values, status=None)

Wait until all variables satisfy the specified conditions.

Blocks until all values specified in ivars not excluded by status satisfy the condition ivars[i] cmp values[i] at the calling PE, where cmp is the comparison operator.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be compared.

• cmp (CMP) – The comparison operator that compares elements of ivars with the elements of values.

• values (Sequence[int | float | complex | number]) – Local array containing values to be compared with the respective elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the wait set. Nonzero values exclude the corresponding element from the wait set.

Return type:

None

shmem4py.shmem.wait_until_any_vector(ivars, cmp, values, status=None)

Wait until any one variable satisfies the specified conditions.

Blocks until any one value specified in ivars not excluded by status satisfies the condition ivars[i] cmp values[i] at the calling PE, where cmp is the comparison operator.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be compared.

• cmp (CMP) – The comparison operator that compares elements of ivars with the elements of values.

• values (Sequence[int | float | complex | number]) – Local array containing values to be compared with the respective elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the wait set. Nonzero values exclude the corresponding element from the wait set.

Returns:

The index of entry i of ivars that satisfies the condition.

Return type:

int | None

shmem4py.shmem.wait_until_some_vector(ivars, cmp, values, status=None)

Wait until at least one variable satisfies the specified conditions.

Blocks until any one value specified in ivars not excluded by status satisfies the condition ivars[i] cmp values[i] at the calling PE, where cmp is the comparison operator.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be compared.

• cmp (CMP) – The comparison operator that compares elements of ivars with the elements of values.

• values (Sequence[int | float | complex | number]) – Local array containing values to be compared with the respective elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the wait set. Nonzero values exclude the corresponding element from the wait set.

Returns:

Indices of entries of ivars that satisfy the condition.

Return type:

List[int]

shmem4py.shmem.test(ivar, cmp, value)

Indicate whether a variable on the local PE meets a condition.

Parameters:

• ivar (NDArray[Any]) – Symmetric array of size 1 containing the element that will be tested.

• cmp (CMP) – The comparison operator that compares ivar with value.

• value (int | float | complex | number) – The value to be compared with ivar.

Return type:

bool

shmem4py.shmem.test_all(ivars, cmp, value, status=None)

Indicate whether all variables on the local PE meet a condition.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be tested.

• cmp (CMP) – The comparison operator that compares elements of ivars with value.

• value (int | float | complex | number) – The value to be compared with elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the test set. Nonzero values exclude the corresponding element from the test set.

Return type:

bool

shmem4py.shmem.test_any(ivars, cmp, value, status=None)

Indicate whether any one variable on the local PE meets a condition.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be tested.

• cmp (CMP) – The comparison operator that compares elements of ivars with value.

• value (int | float | complex | number) – The value to be compared with elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the test set. Nonzero values exclude the corresponding element from the test set.

Returns:

The index of entry i of ivars that satisfies the condition.

Return type:

int | None

shmem4py.shmem.test_some(ivars, cmp, value, status=None)

Indicate whether at least one variable on the local PE meets a condition.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be tested.

• cmp (CMP) – The comparison operator that compares elements of ivars with value.

• value (int | float | complex | number) – The value to be compared with elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the test set. Nonzero values exclude the corresponding element from the test set.

Returns:

Indices of entries of ivars that satisfy the condition.

Return type:

List[int]

shmem4py.shmem.test_all_vector(ivars, cmp, values, status=None)

Indicate whether all variables on the local PE meet the specified conditions.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be tested.

• cmp (CMP) – The comparison operator that compares elements of ivars with the elements of values.

• values (Sequence[int | float | complex | number]) – Local array containing values to be compared with the respective elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the test set. Nonzero values exclude the corresponding element from the test set.

Return type:

bool

shmem4py.shmem.test_any_vector(ivars, cmp, values, status=None)

Indicate whether any one variable on the local PE meets its specified condition.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be tested.

• cmp (CMP) – The comparison operator that compares elements of ivars with the elements of values.

• values (Sequence[int | float | complex | number]) – Local array containing values to be compared with the respective elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the test set. Nonzero values exclude the corresponding element from the test set.

Returns:

The index of entry i of ivars that satisfies the condition.

Return type:

int | None

shmem4py.shmem.test_some_vector(ivars, cmp, values, status=None)

Indicate whether at least one variable on the local PE meets its specified condition.

Parameters:

• ivars (NDArray[Any]) – Symmetric array of objects to be tested.

• cmp (CMP) – The comparison operator that compares elements of ivars with the elements of values.

• values (Sequence[int | float | complex | number]) – Local array containing values to be compared with the respective elements of ivars.

• status (Sequence[int] | None) – An optional mask array of length len(ivars) indicating which elements of ivars are excluded from the test set. Nonzero values exclude the corresponding element from the test set.

Returns:

Indices of entries of ivars that satisfy the condition.

Return type:

List[int]

shmem4py.shmem.signal_wait_until(signal, cmp, value)

Wait for a variable on the local PE to change from a signaling operation.

Parameters:

• signal (SigAddr) – Local symmetric source signal variable.

• cmp (CMP) – The comparison operator that compares signal with value.

• value (int | float | complex | number) – The value against which the object pointed to by signal will be compared.

Returns:

The contents of the signal data object, signal, at the calling PE that satisfies the wait condition.

Return type:

int

class shmem4py.shmem.CMP(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Comparison operator.

EQ

Equal to.

Type:

int

NE

Not equal to.

Type:

int

GT

Greater than.

Type:

int

LE

Less than or equal to.

Type:

int

LT

Less than.

Type:

int

GE

Greater than or equal to.

Type:

int

Memory Ordering

fence([ctx])	Ensure ordering of delivery of operations on symmetric data objects.
quiet([ctx])	Wait for completion of outstanding operations on symmetric data objects issued by a PE.

shmem4py.shmem.fence(ctx=None)

Ensure ordering of delivery of operations on symmetric data objects.

All operations on symmetric data objects issued to a particular PE on the given context prior to the call to fence are guaranteed to be delivered before any subsequent operations on symmetric data objects to the same PE.

Parameters:

ctx (Ctx | None) – A context handle specifying the context on which to perform the operation. If None, defaults to the default context.

Return type:

None

shmem4py.shmem.quiet(ctx=None)

Wait for completion of outstanding operations on symmetric data objects issued by a PE.

Ensures completion of all operations on symmetric data objects issued by the calling PE on the given context.

Parameters:

ctx (Ctx | None) – A context handle specifying the context on which to perform the operation. If None, defaults to the default context.

Return type:

None

Distributed Locking

new_lock()	Create a lock object.
del_lock(lock)	Delete a lock object.
set_lock(lock)	Acquire a mutual exclusion lock after waiting for the lock to be freed.
test_lock(lock)	Acquire a mutual exclusion lock only if it is currently cleared.
clear_lock(lock)	Release a lock previously set by set_lock or test_lock.

shmem4py.shmem.new_lock()

Create a lock object.

Return type:

LockHandle

shmem4py.shmem.del_lock(lock)

Delete a lock object.

Parameters:

lock (LockHandle) – A lock object to be deleted.

Return type:

None

shmem4py.shmem.set_lock(lock)

Acquire a mutual exclusion lock after waiting for the lock to be freed.

Parameters:

lock (LockHandle) – Symmetric scalar variable or an array of length 1.

Return type:

None

shmem4py.shmem.test_lock(lock)

Acquire a mutual exclusion lock only if it is currently cleared.

By using this routine, a PE can avoid blocking on a set lock.

Parameters:

lock (LockHandle) – Symmetric scalar variable or an array of length 1.

Returns:

Returns False if the lock was originally cleared and this call was able to acquire the lock. True is returned if the lock had been set and the call returned without waiting to set the lock.

Return type:

bool

shmem4py.shmem.clear_lock(lock)

Release a lock previously set by set_lock or test_lock.

Releases a lock after performing a quiet operation on the default context to ensure that all symmetric memory accesses that occurred during the critical region are complete.

Parameters:

lock (LockHandle) – Symmetric scalar variable or an array of length 1.

Return type:

None

Distributed Locking (Object-Oriented)

Lock()	Lock object.
Lock.destroy()	Destroy the lock object.
Lock.acquire([blocking])	Acquire the lock.
Lock.release()	Release the lock.

class shmem4py.shmem.Lock

Lock object.

destroy()

Destroy the lock object.

Return type:

None

acquire(blocking=True)

Acquire the lock.

Parameters:

blocking (bool) – True to wait until the lock is acquired.

Returns:

If blocking is True, waits and returns True once the lock has been acquired. If blocking is False, returns True if the lock has been acquired and False otherwise (i.e., lock was already set).

Return type:

bool

release()

Release the lock.

Releases a lock after performing a quiet operation on the default context to ensure that all symmetric memory accesses that occurred during the critical region are complete.

Return type:

None

Profiling Control

pcontrol([level])

Set the profiling level.

shmem4py.shmem.pcontrol(level=1)

Set the profiling level.

Parameters:

level (int) – The profiling level.

Return type:

None

Typing Support

shmem4py.shmem.Number

Numeric type.

alias of int | float | complex | number

shmem4py.shmem.SigAddr = shmem4py.shmem.SigAddr

Signal address.

shmem4py.shmem.CtxHandle = shmem4py.shmem.CtxHandle

Context handle.

shmem4py.shmem.TeamHandle = shmem4py.shmem.TeamHandle

Team handle.

shmem4py.shmem.LockHandle = shmem4py.shmem.LockHandle

Lock handle.

ffi.CData

See ffi.CData