arkouda.numpy.util

Functions

attach(name)	Attach a previously created Arkouda object by its registered name.
attach_all(names)	Attach to all objects registered with the provided names.
broadcast_arrays(...)	Broadcast arrays to a common shape.
broadcast_dims(→ Tuple[int, Ellipsis])	Determine the broadcasted shape of two arrays given their shapes.
broadcast_shapes(→ Tuple[int, Ellipsis])	Determine a broadcasted shape, given an arbitary number of shapes.
convert_bytes(→ arkouda.numpy.dtypes.numeric_scalars)	Convert a number of bytes to a larger unit: KB, MB, or GB.
convert_if_categorical(values)	Convert a Categorical array to a Strings array for display purposes.
copy(→ Union[arkouda.numpy.strings.Strings, ...)	Return a deep copy of the given Arkouda object.
generic_concat(items[, ordered])
get_callback(x)
identity(x)
invert_permutation(→ arkouda.numpy.pdarrayclass.pdarray)	Compute the inverse of a permutation array.
is_float(→ bool)	Check if the dtype of the given array-like object is a float type.
is_int(→ bool)	Check if the dtype of the given array-like object is an integer type.
is_numeric(→ bool)	Check if the dtype of the given array-like object is numeric.
is_registered(→ bool)	Determine whether the provided name is associated with a registered Arkouda object.
map(→ Union[arkouda.numpy.pdarrayclass.pdarray, ...)	Map the values of an array according to an input mapping.
may_share_memory(a, b)	Conservative version akin to numpy.may_share_memory.
register(obj, name)	Register an Arkouda object with a user-specified name.
register_all(data)	Register all objects in the provided dictionary.
report_mem([pre])
shares_memory(a, b)	Return True if a and b share any Arkouda server-side buffers.
sparse_sum_help(...)	Sum two sparse matrices together.
unregister(→ str)	Unregister an Arkouda object by its name.
unregister_all(names)	Unregister all Arkouda objects associated with the provided names.

Module Contents

arkouda.numpy.util.attach(name: str)

Attach a previously created Arkouda object by its registered name.

This function retrieves an Arkouda object (e.g., pdarray, DataFrame, Series) associated with a given name. The returned object type depends on the object stored under that name.

Parameters:

name (str) – The name of the object to attach.

Returns:

The Arkouda object associated with the given name. The returned object could be any supported type, such as pdarray, DataFrame, or Series.

Return type:

object

Raises:

ValueError – Raised if the object type in the response message does not match any known Arkouda types.

Examples

Attach an existing pdarray:

>>> import arkouda as ak
>>> obj = ak.array([1, 2, 3])
>>> registered_obj = obj.register("my_array")
>>> arr = ak.attach("my_array")
>>> print(arr)
[1 2 3]
>>> registered_obj.unregister()

arkouda.numpy.util.attach_all(names: list)

Attach to all objects registered with the provided names.

This function returns a dictionary mapping each name in the input list to the corresponding Arkouda object retrieved using attach.

Parameters:

names (list of str) – A list of names corresponding to registered Arkouda objects.

Returns:

A dictionary mapping each name to the attached Arkouda object.

Return type:

dict

Examples

>>> import arkouda as ak
>>> data = {"arr1": ak.array([0, 1, 2]), "arr2": ak.array([3, 4, 5])}
>>> ak.register_all(data)

Assuming "arr1" and "arr2" were previously registered:

>>> attached_objs = ak.attach_all(["arr1", "arr2"])
>>> print(attached_objs["arr1"])
[0 1 2]
>>> print(type(attached_objs["arr2"]))
<class 'arkouda.numpy.pdarrayclass.pdarray'>

>>> ak.unregister_all(["arr1", "arr2"])

arkouda.numpy.util.broadcast_arrays(*arrays: arkouda.numpy.pdarrayclass.pdarray) → List[arkouda.numpy.pdarrayclass.pdarray]

Broadcast arrays to a common shape.

Parameters:

arrays (pdarray) – The arrays to broadcast. Must be broadcastable to a common shape.

Returns:

A list whose elements are the given Arrays broadcasted to the common shape.

Return type:

List

Raises:

ValueError – Raised by broadcast_to if a common shape cannot be determined.

Examples

>>> import arkouda as ak
>>> a = ak.arange(10).reshape(1,2,5)
>>> b = ak.arange(20).reshape(4,1,5)
>>> c = ak.broadcast_arrays(a,b)
>>> c[0][0,:,:]
array([array([0 1 2 3 4]) array([5 6 7 8 9])])
>>> c[1][:,0,0]
array([0 5 10 15])

arkouda.numpy.util.broadcast_dims(sa: Sequence[int], sb: Sequence[int]) → Tuple[int, Ellipsis]

Determine the broadcasted shape of two arrays given their shapes.

This function implements the broadcasting rules from the Array API standard to compute the shape resulting from broadcasting two arrays together.

See: https://data-apis.org/array-api/latest/API_specification/broadcasting.html#algorithm

Parameters:

• sa (Sequence[int]) – The shape of the first array.

• sb (Sequence[int]) – The shape of the second array.

Returns:

The broadcasted shape resulting from combining sa and sb.

Return type:

Tuple[int, …]

Raises:

ValueError – If the shapes are not compatible for broadcasting.

Examples

>>> import arkouda as ak
>>> from arkouda.numpy.util import broadcast_dims
>>> broadcast_dims((5, 1), (1, 3))
(5, 3)

>>> broadcast_dims((4,), (3, 1))
(3, 4)

arkouda.numpy.util.broadcast_shapes(*shapes: Tuple[int, Ellipsis]) → Tuple[int, Ellipsis]

Determine a broadcasted shape, given an arbitary number of shapes.

This function implements the broadcasting rules from the Array API standard to compute the shape resulting from broadcasting two arrays together.

See: https://data-apis.org/array-api/latest/API_specification/broadcasting.html#algorithm

Parameters:

shapes (Tuple[int, ...]) – a list or tuple of the shapes to be broadcast

Returns:

The broadcasted shape

Return type:

Tuple[int, …]

Raises:

ValueError – If the shapes are not compatible for broadcasting.

Examples

>>> import arkouda as ak
>>> ak.broadcast_shapes((1,2,3),(4,1,3),(4,2,1))
(4, 2, 3)

arkouda.numpy.util.convert_bytes(nbytes: arkouda.numpy.dtypes.int_scalars, unit: Literal['B', 'KB', 'MB', 'GB'] = 'B') → arkouda.numpy.dtypes.numeric_scalars

Convert a number of bytes to a larger unit: KB, MB, or GB.

Parameters:

• nbytes (int_scalars) – The number of bytes to convert.

• unit ({"B", "KB", "MB", "GB"}, default="B") – The unit to convert to. One of {“B”, “KB”, “MB”, “GB”}.

Returns:

The converted value in the specified unit.

Return type:

numeric_scalars

Raises:

ValueError – If unit is not one of {“B”, “KB”, “MB”, “GB”}.

Examples

>>> import arkouda as ak
>>> from arkouda.numpy.util import convert_bytes
>>> convert_bytes(2048, unit="KB")
2.0

>>> convert_bytes(1048576, unit="MB")
1.0

>>> convert_bytes(1073741824, unit="GB")
1.0

arkouda.numpy.util.convert_if_categorical(values)

Convert a Categorical array to a Strings array for display purposes.

If the input is a Categorical, it is converted to its string labels based on its codes. Otherwise, the input is returned unchanged.

Parameters:

values (Categorical or any) – The input array, which may be a Categorical.

Returns:

The string labels if values is a Categorical; otherwise the original input.

Return type:

Strings or any

Examples

Convert a Categorical to its string labels:

>>> import arkouda as ak
>>> categories = ak.array(["apple", "banana", "cherry"])
>>> cat = ak.Categorical(categories)
>>> result = convert_if_categorical(cat)
>>> print(result)
['apple', 'banana', 'cherry']

Non-Categorical inputs are returned unchanged:

>>> values = ak.array([1, 2, 3])
>>> result = convert_if_categorical(values)
>>> print(result)
[1 2 3]

arkouda.numpy.util.copy(a: arkouda.numpy.strings.Strings | arkouda.numpy.pdarrayclass.pdarray) → arkouda.numpy.strings.Strings | arkouda.numpy.pdarrayclass.pdarray

Return a deep copy of the given Arkouda object.

Parameters:

a (Union[Strings, pdarray]) – The object to copy.

Returns:

A deep copy of the pdarray or Strings object.

Return type:

Union[Strings, pdarray]

Raises:

TypeError – If the input is not a Strings or pdarray instance.

arkouda.numpy.util.generic_concat(items, ordered=True)

arkouda.numpy.util.get_callback(x)

arkouda.numpy.util.identity(x)

arkouda.numpy.util.invert_permutation(perm: arkouda.numpy.pdarrayclass.pdarray) → arkouda.numpy.pdarrayclass.pdarray

Compute the inverse of a permutation array.

The inverse permutation undoes the effect of the original permutation. For a valid permutation array perm, this function returns an array inv such that inv[perm[i]] == i for all i.

Parameters:

perm (pdarray) – A permutation of the integers [0, N-1], where N is the length of the array.

Returns:

The inverse of the input permutation.

Return type:

pdarray

Raises:

ValueError – If perm is not a valid permutation of the range [0, N-1], such as containing duplicates or missing values.

Examples

>>> import arkouda as ak
>>> from arkouda import array, invert_permutation
>>> perm = array([2, 0, 3, 1])
>>> inv = invert_permutation(perm)
>>> print(inv)
[1 3 0 2]

arkouda.numpy.util.is_float(arry: arkouda.numpy.pdarrayclass.pdarray | arkouda.numpy.strings.Strings | arkouda.pandas.categorical.Categorical | arkouda.pandas.series.Series | arkouda.pandas.index.Index) → bool

Check if the dtype of the given array-like object is a float type.

Parameters:

arry (pdarray, Strings, Categorical, Series, or Index) – The object to check.

Returns:

True if the dtype of arry is a float type, False otherwise.

Return type:

bool

Examples

>>> import arkouda as ak
>>> data = ak.array([1.0, 2, 3, 4, float('nan')])
>>> ak.util.is_float(data)
True

>>> data2 = ak.arange(5)
>>> ak.util.is_float(data2)
False

>>> strings = ak.array(["1.0", "2.0"])
>>> ak.util.is_float(strings)
False

arkouda.numpy.util.is_int(arry: arkouda.numpy.pdarrayclass.pdarray | arkouda.numpy.strings.Strings | arkouda.pandas.categorical.Categorical | arkouda.pandas.series.Series | arkouda.pandas.index.Index) → bool

Check if the dtype of the given array-like object is an integer type.

Parameters:

arry (pdarray, Strings, Categorical, Series, or Index) – The object to check.

Returns:

True if the dtype of arry is an integer type, False otherwise.

Return type:

bool

Examples

>>> import arkouda as ak
>>> data = ak.array([1.0, 2, 3, 4, float('nan')])
>>> ak.util.is_int(data)
False

>>> data2 = ak.arange(5)
>>> ak.util.is_int(data2)
True

>>> strings = ak.array(["1", "2"])
>>> ak.util.is_int(strings)
False

arkouda.numpy.util.is_numeric(arry: arkouda.numpy.pdarrayclass.pdarray | arkouda.numpy.strings.Strings | arkouda.pandas.categorical.Categorical | arkouda.pandas.series.Series | arkouda.pandas.index.Index) → bool

Check if the dtype of the given array-like object is numeric.

Parameters:

arry (pdarray, Strings, Categorical, Series, or Index) – The object to check.

Returns:

True if the dtype of arry is numeric, False otherwise.

Return type:

bool

Examples

>>> import arkouda as ak
>>> data = ak.array([1, 2, 3, 4, 5])
>>> ak.util.is_numeric(data)
True

>>> strings = ak.array(["a", "b", "c"])
>>> ak.util.is_numeric(strings)
False

>>> from arkouda import Categorical
>>> cat = Categorical(strings)
>>> ak.util.is_numeric(cat)
False

arkouda.numpy.util.is_registered(name: str, as_component: bool = False) → bool

Determine whether the provided name is associated with a registered Arkouda object.

This function checks whether name is found in the registry of objects and optionally checks whether it is registered as a component of a registered object.

Parameters:

• name (str) – The name to check in the registry.

• as_component (bool, default=False) – When True, the function checks whether the name is registered as a component of a registered object rather than as a standalone object.

Returns:

True if the name is found in the registry, False otherwise.

Return type:

bool

Raises:

KeyError – Raised if the registry query encounters an issue (e.g., invalid registry data or access problems).

Examples

Check whether a name is registered as an object:

>>> import arkouda as ak
>>> obj = ak.array([1, 2, 3])
>>> registered_obj = obj.register("my_array")
>>> result = ak.is_registered("my_array")
>>> print(result)
True
>>> registered_obj.unregister()

Check whether a name is registered as a component:

>>> result = ak.is_registered("my_component", as_component=True)
>>> print(result)
False

arkouda.numpy.util.map(values: arkouda.numpy.pdarrayclass.pdarray | arkouda.numpy.strings.Strings | arkouda.pandas.categorical.Categorical, mapping: dict | arkouda.Series) → arkouda.numpy.pdarrayclass.pdarray | arkouda.numpy.strings.Strings

Map the values of an array according to an input mapping.

Parameters:

• values (pdarray, Strings, or Categorical) – The values to be mapped.

• mapping (dict or Series) – The mapping correspondence. A dictionary or Series that defines how to map the values array.

Returns:

A new array with the values mapped by the provided mapping. The return type matches the type of values. If the input Series has Categorical values, the return type will be Strings.

Return type:

Union[pdarray, Strings]

Raises:

• TypeError – If mapping is not of type dict or Series. If values is not of type pdarray, Categorical, or Strings.

• ValueError – If a mapping with tuple keys has inconsistent lengths, or if a MultiIndex mapping has a different number of levels than the GroupBy keys.

Examples

>>> import arkouda as ak
>>> from arkouda.numpy.util import map
>>> a = ak.array([2, 3, 2, 3, 4])
>>> a
array([2 3 2 3 4])
>>> ak.util.map(a, {4: 25.0, 2: 30.0, 1: 7.0, 3: 5.0})
array([30.00000000000000000 5.00000000000000000 30.00000000000000000
5.00000000000000000 25.00000000000000000])
>>> s = ak.Series(ak.array(["a", "b", "c", "d"]), index=ak.array([4, 2, 1, 3]))
>>> ak.util.map(a, s)
array(['b', 'd', 'b', 'd', 'a'])

arkouda.numpy.util.may_share_memory(a, b)

Conservative version akin to numpy.may_share_memory.

For now it just defers to shares_memory.

arkouda.numpy.util.register(obj, name)

Register an Arkouda object with a user-specified name.

This function registers the provided Arkouda object (obj) under a given name (name). It is maintained for backwards compatibility with earlier versions of Arkouda.

Parameters:

• obj (Arkouda object) – The Arkouda object to register.

• name (str) – The name to associate with the object.

Returns:

The input object, now registered with the specified name.

Return type:

object

Raises:

AttributeError – Raised if obj does not have a register method.

Examples

>>> import arkouda as ak
>>> from arkouda.numpy.util import register
>>> obj = ak.array([1, 2, 3])
>>> registered_obj = register(obj, "my_array")
>>> print(registered_obj)
[1 2 3]
>>> registered_obj.unregister()

Register a different Arkouda object:

>>> categories = ak.array(["apple", "banana", "cherry"])
>>> cat = ak.Categorical(categories)
>>> registered_cat = register(cat, "my_cat")
>>> print(registered_cat)
['apple', 'banana', 'cherry']

arkouda.numpy.util.register_all(data: dict)

Register all objects in the provided dictionary.

This function iterates through the dictionary data, registering each object with its corresponding name. It is useful for batch-registering multiple objects in Arkouda.

Parameters:

data (dict) – A dictionary mapping the name used to register the object to the object itself. For example, {"MyArray": ak.array([0, 1, 2])}.

Examples

>>> import arkouda as ak
>>> data = {"array1": ak.array([0, 1, 2]), "array2": ak.array([3, 4, 5])}
>>> ak.register_all(data)

After calling this function, "array1" and "array2" are registered in Arkouda and can be accessed by their names.

>>> ak.unregister_all(["array1", "array2"])

arkouda.numpy.util.report_mem(pre='')

arkouda.numpy.util.shares_memory(a, b)

Return True if a and b share any Arkouda server-side buffers.

This is an Arkouda analogue of numpy.shares_memory with a simpler definition: it checks for identical backing buffer identities (same server object names).

Notes

• Because Arkouda commonly materializes results (rather than views), aliasing is rare and usually only true when objects literally reference the same backing buffers.

• For compound containers (e.g., SegArray, Strings, Categorical), we check all of their component buffers.

• If you introduce true view semantics in the future, teach _ak_buffer_names to surface the base buffer name(s) and view descriptors, and compare bases.

arkouda.numpy.util.sparse_sum_help(idx1: arkouda.numpy.pdarrayclass.pdarray, idx2: arkouda.numpy.pdarrayclass.pdarray, val1: arkouda.numpy.pdarrayclass.pdarray, val2: arkouda.numpy.pdarrayclass.pdarray, merge: bool = True, percent_transfer_limit: int = 100) → Tuple[arkouda.numpy.pdarrayclass.pdarray, arkouda.numpy.pdarrayclass.pdarray]

Sum two sparse matrices together.

This function returns the result of summing two sparse matrices by combining their indices and values. Internally, it performs the equivalent of:

ak.GroupBy(ak.concatenate([idx1, idx2])).sum(ak.concatenate((val1, val2)))

Parameters:

• idx1 (pdarray) – Indices for the first sparse matrix.

• idx2 (pdarray) – Indices for the second sparse matrix.

• val1 (pdarray) – Values for the first sparse matrix.

• val2 (pdarray) – Values for the second sparse matrix.

• merge (bool, default=True) – If True, the indices are combined using a merge-based workflow. If False, a sort-based workflow is used.

• percent_transfer_limit (int, default=100) – Only used when merge is True. This defines the maximum percentage of data allowed to move between locales during the merge. If this threshold is exceeded, a sort-based workflow is used instead.

Returns:

A tuple containing: - The indices of the resulting sparse matrix. - The summed values associated with those indices.

Return type:

Tuple[pdarray, pdarray]

Examples

>>> import arkouda as ak
>>> idx1 = ak.array([0, 1, 3, 4, 7, 9])
>>> idx2 = ak.array([0, 1, 3, 6, 9])
>>> vals1 = idx1
>>> vals2 = ak.array([10, 11, 13, 16, 19])
>>> ak.util.sparse_sum_help(idx1, idx2, vals1, vals2)
(array([0 1 3 4 6 7 9]), array([10 12 16 4 16 7 28]))

>>> ak.GroupBy(ak.concatenate([idx1, idx2])).sum(ak.concatenate((vals1, vals2)))
(array([0 1 3 4 6 7 9]), array([10 12 16 4 16 7 28]))

arkouda.numpy.util.unregister(name: str) → str

Unregister an Arkouda object by its name.

This function sends a request to unregister the Arkouda object associated with the specified name. It returns a response message indicating the success or failure of the operation.

Parameters:

name (str) – The name of the object to unregister.

Returns:

A message indicating the result of the unregister operation.

Return type:

str

Raises:

RuntimeError – If the object associated with the given name does not exist or cannot be unregistered.

Examples

>>> import arkouda as ak

Unregister an existing object >>> obj = ak.array([1, 2, 3]) >>> registered_obj = obj.register(“my_array”) >>> response = ak.unregister(“my_array”) >>> print(response) Unregistered PDARRAY my_array

arkouda.numpy.util.unregister_all(names: List[str])

Unregister all Arkouda objects associated with the provided names.

This function iterates through the list of names, unregistering each corresponding object from the Arkouda server.

Parameters:

names (list of str) – A list of registered names corresponding to Arkouda objects that should be unregistered.

Examples

>>> import arkouda as ak
>>> data = {"array1": ak.array([0, 1, 2]), "array2": ak.array([3, 4, 5])}
>>> ak.register_all(data)

After calling this function, "array1" and "array2" are registered in Arkouda and can be accessed by their names.

>>> ak.unregister_all(["array1", "array2"])

The objects are now unregistered.