arkouda.matcher¶

Attributes¶

`MatchType`
`generic_msg`

Classes¶

`Match`
`Matcher`
`pdarray`	The basic arkouda array class. This class contains only the
`str_scalars`	Mixin to prevent iteration, without being compatible with Iterable.

Functions¶

`create_pdarray`(→ pdarray)	Return a pdarray instance pointing to an array created by the arkouda server.
`getArkoudaLogger`(→ ArkoudaLogger)	Instantiate an ArkoudaLogger that retrieves the logging level from ARKOUDA_LOG_LEVEL env variable.
`list_symbol_table`(→ List[str])	Return a list containing the names of all objects in the symbol table.

Package Contents¶

class arkouda.matcher.Match(matched: arkouda.numpy.pdarrayclass.pdarray, starts: arkouda.numpy.pdarrayclass.pdarray, lengths: arkouda.numpy.pdarrayclass.pdarray, indices: arkouda.numpy.pdarrayclass.pdarray, parent_entry_name: str, match_type: MatchType, pattern: str)[source]¶

end() → arkouda.numpy.pdarrayclass.pdarray[source]¶

Return the ends of matches.

Returns:: The end positions of matches
Return type:: pdarray

Examples

>>> import arkouda as ak
>>> strings = ak.array(['1_2___', '____', '3', '__4___5____6___7', ''])
>>> strings.search('_+').end()
array([2 4 2])

find_matches(return_match_origins: bool = False)[source]¶

Return all matches as a new Strings object.

Parameters:

return_match_origins (bool) – If True, return a pdarray containing the index of the original string each pattern match is from

Returns:

Strings – Strings object containing only matches
pdarray, int64 (optional) – The index of the original string each pattern match is from

Raises:

RuntimeError – Raised if there is a server-side error thrown

Examples

>>> import arkouda as ak
>>> strings = ak.array(['1_2___', '____', '3', '__4___5____6___7', ''])
>>> strings.search('_+').find_matches(return_match_origins=True)
(array(['_', '____', '__']), array([0 1 3]))

group(group_num: int = 0, return_group_origins: bool = False)[source]¶

Return a new Strings containing the capture group corresponding to group_num.

For the default, group_num=0, return the full match.

Parameters:

group_num (int) – The index of the capture group to be returned
return_group_origins (bool) – If True, return a pdarray containing the index of the original string each capture group is from

Returns:

Strings – Strings object containing only the capture groups corresponding to group_num
pdarray, int64 (optional) – The index of the original string each group is from

Examples

>>> import arkouda as ak
>>> strings = ak.array(["Isaac Newton, physics", '<-calculus->', 'Gottfried Leibniz, math'])
>>> m = strings.search("(\\w+) (\\w+)")
>>> m.group()
array(['Isaac Newton', 'Gottfried Leibniz'])
>>> m.group(1)
array(['Isaac', 'Gottfried'])
>>> m.group(2, return_group_origins=True)
(array(['Newton', 'Leibniz']), array([0 2]))

match_type() → str[source]¶

Return the type of the Match object.

Returns:: MatchType of the Match object
Return type:: str

Examples

>>> import arkouda as ak
>>> strings = ak.array(['1_2___', '____', '3', '__4___5____6___7', ''])
>>> strings.search('_+').match_type()
'SEARCH'

matched() → arkouda.numpy.pdarrayclass.pdarray[source]¶

Return a boolean array indiciating whether each element matched.

Returns:: True for elements that match, False otherwise
Return type:: pdarray

Examples

>>> import arkouda as ak
>>> strings = ak.array(['1_2___', '____', '3', '__4___5____6___7', ''])
>>> strings.search('_+').matched()
array([True True False True False])

re¶

start() → arkouda.numpy.pdarrayclass.pdarray[source]¶

Return the starts of matches.

Returns:: The start positions of matches
Return type:: pdarray

Examples

>>> import arkouda as ak
>>> strings = ak.array(['1_2___', '____', '3', '__4___5____6___7', ''])
>>> strings.search('_+').start()
array([1 0 0])

arkouda.matcher.MatchType¶

class arkouda.matcher.Matcher(pattern: arkouda.numpy.dtypes.str_scalars, parent_entry_name: str)[source]¶

LocationsInfo¶

find_locations() → None[source]¶: Populate Matcher object by finding the positions of matches.

findall(return_match_origins: bool = False)[source]¶: Return all non-overlapping matches of pattern in Strings as a new Strings object.

full_match_bool: arkouda.numpy.pdarrayclass.pdarray¶

full_match_ind: arkouda.numpy.pdarrayclass.pdarray¶

get_match(match_type: arkouda.pandas.match.MatchType, parent: object = None) → arkouda.pandas.match.Match[source]¶: Create a Match object of type match_type.

indices: arkouda.numpy.pdarrayclass.pdarray¶

lengths: arkouda.numpy.pdarrayclass.pdarray¶

logger¶

match_bool: arkouda.numpy.pdarrayclass.pdarray¶

match_ind: arkouda.numpy.pdarrayclass.pdarray¶

num_matches: arkouda.numpy.pdarrayclass.pdarray¶

objType = 'Matcher'¶

parent_entry_name¶

populated = False¶

search_bool: arkouda.numpy.pdarrayclass.pdarray¶

search_ind: arkouda.numpy.pdarrayclass.pdarray¶

split(maxsplit: int = 0, return_segments: bool = False)[source]¶: Split string by the occurrences of pattern. If maxsplit is nonzero, at most maxsplit splits occur.

starts: arkouda.numpy.pdarrayclass.pdarray¶

sub(repl: str, count: int = 0, return_num_subs: bool = False)[source]¶: Return the Strings obtained by replacing non-overlapping occurrences of pattern with the replacement repl. If count is nonzero, at most count substitutions occur If return_num_subs is True, return the number of substitutions that occurred

arkouda.matcher.create_pdarray(repMsg: str, max_bits=None) → pdarray[source]¶

Return a pdarray instance pointing to an array created by the arkouda server. The user should not call this function directly.

Parameters:

repMsg (str) – space-delimited string containing the pdarray name, datatype, size dimension, shape,and itemsize

Returns:

A pdarray with the same attributes and data as the pdarray; on GPU

Return type:

pdarray

Raises:

ValueError – If there’s an error in parsing the repMsg parameter into the six values needed to create the pdarray instance
RuntimeError – Raised if a server-side error is thrown in the process of creating the pdarray instance

arkouda.matcher.generic_msg¶

arkouda.matcher.getArkoudaLogger(name: str, handlers: List[logging.Handler] | None = None, logFormat: str | None = ArkoudaLogger.DEFAULT_LOG_FORMAT, logLevel: LogLevel | None = None) → ArkoudaLogger[source]¶

Instantiate an ArkoudaLogger that retrieves the logging level from ARKOUDA_LOG_LEVEL env variable.

Parameters:

name (str) – The name of the ArkoudaLogger
handlers (List[Handler]) – A list of logging.Handler objects, if None, a list consisting of one StreamHandler named ‘console-handler’ is generated and configured
logFormat (str) – The format for log messages, defaults to the following format: ‘[%(name)s] Line %(lineno)d %(levelname)s: %(message)s’

Return type:

ArkoudaLogger

Raises:

TypeError – Raised if either name or logFormat is not a str object or if handlers is not a list of str objects

Notes

Important note: if a list of 1..n logging.Handler objects is passed in, and dynamic changes to 1..n handlers is desired, set a name for each Handler object as follows: handler.name = <desired name>, which will enable retrieval and updates for the specified handler.

arkouda.matcher.list_symbol_table() → List[str][source]¶

Return a list containing the names of all objects in the symbol table.

Returns:: List of all object names in the symbol table
Return type:: list
Raises:: RuntimeError – Raised if there’s a server-side error thrown

class arkouda.matcher.pdarray(name: str, mydtype: numpy.dtype, size: arkouda.numpy.dtypes.int_scalars, ndim: arkouda.numpy.dtypes.int_scalars, shape: Tuple[int, Ellipsis], itemsize: arkouda.numpy.dtypes.int_scalars, max_bits: int | None = None)[source]¶

The basic arkouda array class. This class contains only the attributes of the array; the data resides on the arkouda server. When a server operation results in a new array, arkouda will create a pdarray instance that points to the array data on the server. As such, the user should not initialize pdarray instances directly.

name¶

The server-side identifier for the array

Type:: str

dtype¶

The element dtype of the array

Type:: type

size¶

The number of elements in the array

Type:: int_scalars

ndim¶

The rank of the array

Type:: int_scalars

shape¶

A tuple containing the sizes of each dimension of the array

Type:: Tuple[int, …]

itemsize¶

The size in bytes of each element

Type:: int_scalars

BinOps¶

OpEqOps¶

all(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.bool_scalars | pdarray[source]¶

Return True iff all elements of the array along the given axis evaluate to True.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

boolean if axis is omitted, pdarray if axis is supplied

Return type:

boolean or pdarray

Examples

>>> import arkouda as ak
>>> ak.all(ak.array([True,False,False]))
np.False_
>>> ak.all(ak.array([[True,True,False],[False,True,True]]),axis=0)
array([False True False])
>>> ak.all(ak.array([[True,True,True],[False,False,False]]),axis=0,keepdims=True)
array([array([False False False])])
>>> ak.all(ak.array([[True,True,True],[False,False,False]]),axis=1,keepdims=True)
array([array([True]) array([False])])
>>> ak.array([True,False,False]).all()
np.False_

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Notes

Works as a method of a pdarray (e.g. a.any()) or a standalone function (e.g. ak.all(a))

any(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.bool_scalars | pdarray[source]¶

Return True iff any element of the array along the given axis evaluates to True.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

boolean if axis is omitted, else pdarray if axis is supplied

Return type:

boolean or pdarray

Examples

>>> import arkouda as ak
>>> ak.any(ak.array([True,False,False]))
np.True_
>>> ak.any(ak.array([[True,True,False],[False,True,True]]),axis=0)
array([True True True])
>>> ak.any(ak.array([[True,True,True],[False,False,False]]),axis=0,keepdims=True)
array([array([True True True])])
>>> ak.any(ak.array([[True,True,True],[False,False,False]]),axis=1,keepdims=True)
array([array([True]) array([False])])
>>> ak.array([True,False,False]).any()
np.True_

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Notes

Works as a method of a pdarray (e.g. a.any()) or a standalone function (e.g. ak.any(a))

argmax(axis: int | None | None = None, keepdims: bool = False) → numpy.int64 | numpy.uint64 | pdarray[source]¶

Return index of the first occurrence of the maximum along the given axis.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

int64 or uint64 if axis is omitted, in which case operation is done over entire array pdarray if axis is supplied, in which case the operation is done along that axis

Return type:

int64, uint64 or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.argmax(ak.array([1,2,3,4,5]))
np.int64(4)
>>> ak.argmax(ak.array([5.5,4.5,3.5,2.5,1.5]))
np.int64(0)
>>> ak.array([[1,2,3],[5,4,3]]).argmax(axis=1)
array([2 0])

Notes

Works as a method of a pdarray (e.g. a.argmax()) or a standalone function (e.g. ak.argmax(a))

argmaxk(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶: Finds the indices corresponding to the k maximum values of an array. See arkouda.argmaxk for details.

argmin(axis: int | None | None = None, keepdims: bool = False) → numpy.int64 | numpy.uint64 | pdarray[source]¶

Return index of the first occurrence of the minimum along the given axis.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

int64 or uint64 if axis is omitted, in which case operation is done over entire array pdarray if axis is supplied, in which case the operation is done along that axis

Return type:

int64, uint64 or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.argmin(ak.array([1,2,3,4,5]))
np.int64(0)
>>> ak.argmin(ak.array([5.5,4.5,3.5,2.5,1.5]))
np.int64(4)
>>> ak.array([[1,2,3],[5,4,3]]).argmin(axis=1)
array([0 2])

Notes

Works as a method of a pdarray (e.g. a.argmin()) or a standalone function (e.g. ak.argmin(a))

argmink(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶: Finds the indices corresponding to the k minimum values of an array. See arkouda.argmink for details.

argsort(algorithm: arkouda.numpy.sorting.SortingAlgorithm = SortingAlgorithm.RadixSortLSD, axis: arkouda.numpy.dtypes.int_scalars = 0, ascending: bool = True) → pdarray[source]¶

Return the permutation that sorts the pdarray.

Parameters:

algorithm (SortingAlgorithm, default SortingAlgorithm.RadixSortLSD) – The algorithm to use for sorting.
axis (int_scalars, default 0) – The axis to sort along. Must be between -1 and the array rank.
ascending (bool, default True) – Whether to sort in ascending order. If False, returns a reversed permutation. Note: ascending=False is only supported for 1D arrays.

Returns:

The indices that would sort the array.

Return type:

pdarray

Examples

>>> import arkouda as ak
>>> a = ak.array([42, 7, 19])
>>> a.argsort()
array([1 2 0])
>>> a[a.argsort()]
array([7 19 42])
>>> a.argsort(ascending=False)
array([0 2 1])

astype(dtype) → pdarray[source]¶

Cast values of pdarray to provided dtype

Parameters:: dtype (np.dtype or str) – Dtype to cast to

Examples

>>> import arkouda as ak
>>> ak.array([1,2,3]).astype(ak.float64)
array([1.00000000000000000 2.00000000000000000 3.00000000000000000])
>>> ak.array([1.5,2.5]).astype(ak.int64)
array([1 2])
>>> ak.array([True,False]).astype(ak.int64)
array([1 0])

Returns:: An arkouda pdarray with values converted to the specified data type
Return type:: ak.pdarray

Notes

This is essentially shorthand for ak.cast(x, ‘<dtype>’) where x is a pdarray.

bigint_to_uint_arrays() → List[pdarray][source]¶

Create a list of uint pdarrays from a bigint pdarray. The first item in return will be the highest 64 bits of the bigint pdarray and the last item will be the lowest 64 bits.

Returns:: A list of uint pdarrays where: The first item in return will be the highest 64 bits of the bigint pdarray and the last item will be the lowest 64 bits.
Return type:: List[pdarrays]
Raises:: RuntimeError – Raised if there is a server-side error thrown

See also

pdarraycreation.bigint_from_uint_arrays

Examples

>>> import arkouda as ak
>>> a = ak.arange(2**64, 2**64 + 5)
>>> a
array([18446744073709551616 18446744073709551617 18446744073709551618
18446744073709551619 18446744073709551620])
>>> a.bigint_to_uint_arrays()
[array([1 1 1 1 1]), array([0 1 2 3 4])]

clz() → pdarray[source]¶: Count the number of leading zeros in each element. See ak.clz.

corr(y: pdarray) → numpy.float64[source]¶: Compute the correlation between self and y using pearson correlation coefficient. See arkouda.corr for details.

cov(y: pdarray) → numpy.float64[source]¶: Compute the covariance between self and y.

ctz() → pdarray[source]¶: Count the number of trailing zeros in each element. See ak.ctz.

dtype: numpy.dtype¶

equals(other) → arkouda.numpy.dtypes.bool_scalars[source]¶

Whether pdarrays are the same size and all entries are equal.

Parameters:: other (object) – object to compare.
Returns:: True if the pdarrays are the same, o.w. False.
Return type:: bool_scalars

Examples

>>> import arkouda as ak
>>> a = ak.array([1, 2, 3])
>>> a_cpy = ak.array([1, 2, 3])
>>> a.equals(a_cpy)
np.True_
>>> a2 = ak.array([1, 2, 5])
>>> a.equals(a2)
np.False_

fill(value: arkouda.numpy.dtypes.numeric_scalars) → None[source]¶

Fill the array (in place) with a constant value.

Parameters:: value (numeric_scalars)
Raises:: TypeError – Raised if value is not an int, int64, float, or float64

flatten()[source]¶

Return a copy of the array collapsed into one dimension.

Return type:: A copy of the input array, flattened to one dimension.

Examples

>>> import arkouda as ak
>>> a = ak.array([[3,2,1],[2,3,1]])
>>> a.flatten()
array([3 2 1 2 3 1])

format_other(other) → str[source]¶

Attempt to cast scalar other to the element dtype of this pdarray, and print the resulting value to a string (e.g. for sending to a server command). The user should not call this function directly.

Parameters:: other (object) – The scalar to be cast to the pdarray.dtype
Return type:: string representation of np.dtype corresponding to the other parameter
Raises:: TypeError – Raised if the other parameter cannot be converted to Numpy dtype

property inferred_type: str | None¶: Return a string of the type inferred from the values.

info() → str[source]¶

Return a JSON formatted string containing information about all components of self.

Returns:: JSON string containing information about all components of self
Return type:: str

is_registered() → numpy.bool_[source]¶

Return True iff the object is contained in the registry

Returns:: Indicates if the object is contained in the registry
Return type:: bool
Raises:: RuntimeError – Raised if there’s a server-side error thrown

Note

This will return True if the object is registered itself or as a component of another object

is_sorted(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.bool_scalars | pdarray[source]¶

Return True iff the array (or given axis of the array) is monotonically non-decreasing.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

boolean if axis is omitted, else pdarray if axis is supplied

Return type:

boolean or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.is_sorted(ak.array([1,2,3,4,5]))
np.True_
>>> ak.is_sorted(ak.array([5,4,3,2,1]))
np.False_
>>> ak.array([[1,2,3],[5,4,3]]).is_sorted(axis=1)
array([True False])

Notes

Works as a method of a pdarray (e.g. a.is_sorted()) or a standalone function (e.g. ak.is_sorted(a))

itemsize: arkouda.numpy.dtypes.int_scalars¶

max(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numeric_scalars | pdarray[source]¶

Return max of array elements along the given axis.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

numeric_scalar if axis is omitted, in which case operation is done over entire array pdarray if axis is supplied, in which case the operation is done along that axis

Return type:

numeric_scalar or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.max(ak.array([1,2,3,4,5]))
np.int64(5)
>>> ak.max(ak.array([5.5,4.5,3.5,2.5,1.5]))
np.float64(5.5)
>>> ak.array([[1,2,3],[5,4,3]]).max(axis=1)
array([3 5])

Notes

Works as a method of a pdarray (e.g. a.max()) or a standalone function (e.g. ak.max(a))

property max_bits¶

maxk(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶: Compute the maximum “k” values. See arkouda.maxk for details.

mean(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numeric_scalars | pdarray[source]¶

Return the mean of the array.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

The mean calculated from the pda sum and size, along the axis/axes if those are given.

Return type:

Union[np.float64, pdarray]

Examples

>>> import arkouda as ak
>>> a = ak.arange(10)
>>> ak.mean(a)
np.float64(4.5)
>>> a.mean()
np.float64(4.5)
>>> a = ak.arange(10).reshape(2,5)
>>> a.mean(axis=0)
array([2.5 3.5 4.5 5.5 6.5])
>>> ak.mean(a,axis=0)
array([2.5 3.5 4.5 5.5 6.5])
>>> a.mean(axis=1)
array([2.00000000000000000 7.00000000000000000])
>>> ak.mean(a,axis=1)
array([2.00000000000000000 7.00000000000000000])

Raises: Raised if pda is not a pdarray instance
RuntimeError: Raised if there’s a server-side error thrown

min(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numeric_scalars | pdarray[source]¶

Return min of array elements along the given axis.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

numeric_scalar if axis is omitted, in which case operation is done over entire array pdarray if axis is supplied, in which case the operation is done along that axis

Return type:

numeric_scalar or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.min(ak.array([1,2,3,4,5]))
np.int64(1)
>>> ak.min(ak.array([5.5,4.5,3.5,2.5,1.5]))
np.float64(1.5)
>>> ak.array([[1,2,3],[5,4,3]]).min(axis=1)
array([1 3])

Notes

Works as a method of a pdarray (e.g. a.min()) or a standalone function (e.g. ak.min(a))

mink(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶: Compute the minimum “k” values. See arkouda.mink for details.

name: str¶

property nbytes¶

The size of the pdarray in bytes.

Returns:: The size of the pdarray in bytes.
Return type:: int

ndim: arkouda.numpy.dtypes.int_scalars¶

objType = 'pdarray'¶

opeq(other, op)[source]¶

parity() → pdarray[source]¶: Find the parity (XOR of all bits) in each element. See ak.parity.

popcount() → pdarray[source]¶: Find the population (number of bits set) in each element. See ak.popcount.

pretty_print_info() → None[source]¶: Print information about all components of self in a human-readable format.

prod(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numeric_scalars | pdarray[source]¶

Return prod of array elements along the given axis.

Parameters:

axis (int, Tuple[int, ...], optional, defalt = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

numeric_scalars if axis is omitted, in which case operation is done over entire array pdarray if axis is supplied, in which case the operation is done along that axis

Return type:

numeric_scalars or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.prod(ak.array([1,2,3,4,5]))
np.int64(120)
>>> ak.prod(ak.array([5.5,4.5,3.5,2.5,1.5]))
np.float64(324.84375)
>>> ak.array([[1,2,3],[5,4,3]]).prod(axis=1)
array([6 60])

Notes

Works as a method of a pdarray (e.g. a.prod()) or a standalone function (e.g. ak.prod(a))

register(user_defined_name: str) → pdarray[source]¶

Register this pdarray with a user defined name in the arkouda server so it can be attached to later using pdarray.attach() This is an in-place operation, registering a pdarray more than once will update the name in the registry and remove the previously registered name. A name can only be registered to one pdarray at a time.

Parameters:

user_defined_name (str) – user defined name array is to be registered under

Returns:

The same pdarray which is now registered with the arkouda server and has an updated name. This is an in-place modification, the original is returned to support a fluid programming style. Please note you cannot register two different pdarrays with the same name.

Return type:

pdarray

Raises:

TypeError – Raised if user_defined_name is not a str
RegistrationError – If the server was unable to register the pdarray with the user_defined_name If the user is attempting to register more than one pdarray with the same name, the former should be unregistered first to free up the registration name.

See also

attach, unregister, is_registered, list_registry, unregister_pdarray_by_name

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> import arkouda as ak
>>> a = ak.zeros(3)
>>> a.register("my_zeros")
array([0.00000000000000000 0.00000000000000000 0.00000000000000000])

potentially disconnect from server and reconnect to server >>> b = ak.attach(“my_zeros”) >>> b.unregister()

registered_name: str | None = None¶

reshape(*shape)[source]¶

Gives a new shape to an array without changing its data.

Parameters:: shape (int, tuple of ints, or pdarray) – The new shape should be compatible with the original shape.
Returns:: a pdarray with the same data, reshaped to the new shape
Return type:: pdarray

Examples

>>> import arkouda as ak
>>> a = ak.array([[3,2,1],[2,3,1]])
>>> a.reshape((3,2))
array([array([3 2]) array([1 2]) array([3 1])])
>>> a.reshape(3,2)
array([array([3 2]) array([1 2]) array([3 1])])
>>> a.reshape((6,1))
array([array([3]) array([2]) array([1]) array([2]) array([3]) array([1])])

Notes

only available as a method, not as a standalone function, i.e., a.reshape(compatibleShape) is valid, but ak.reshape(a,compatibleShape) is not.

rotl(other) → pdarray[source]¶: Rotate bits left by <other>.

rotr(other) → pdarray[source]¶: Rotate bits right by <other>.

property shape¶

Return the shape of an array.

Returns:: The elements of the shape tuple give the lengths of the corresponding array dimensions.
Return type:: tuple of int

size: arkouda.numpy.dtypes.int_scalars¶

slice_bits(low, high) → pdarray[source]¶

Return a pdarray containing only bits from low to high of self.

This is zero indexed and inclusive on both ends, so slicing the bottom 64 bits is pda.slice_bits(0, 63)

Parameters:

low (int) – The lowest bit included in the slice (inclusive) zero indexed, so the first bit is 0
high (int) – The highest bit included in the slice (inclusive)

Returns:

A new pdarray containing the bits of self from low to high

Return type:

pdarray

Raises:

RuntimeError – Raised if there is a server-side error thrown

Examples

>>> import arkouda as ak
>>> p = ak.array([2**65 + (2**64 - 1)])
>>> bin(p[0])
'0b101111111111111111111111111111111111111111111111111111111111111111'
>>> bin(p.slice_bits(64, 65)[0])
'0b10'
>>> a = ak.array([143,15])
>>> a.slice_bits(1,3)
array([7 7])
>>> a.slice_bits(4,9)
array([8 0])
>>> a.slice_bits(1,9)
array([71 7])

std(ddof: arkouda.numpy.dtypes.int_scalars = 0, axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool | None = False) → numpy.float64 | pdarray[source]¶

Return the standard deviation of values in the array. The standard deviation is implemented as the square root of the variance.

Parameters:

ddof (int_scalars) – “Delta Degrees of Freedom” used in calculating std
axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

The scalar standard deviation of the array, or the standard deviation: along the axis/axes if supplied

Return type:

Union[np.float64, pdarray]

Examples

>>> import arkouda as ak
>>> a = ak.arange(10)
>>> ak.std(a)
np.float64(2.8722813232690143)
>>> a.std()
np.float64(2.8722813232690143)
>>> a = ak.arange(10).reshape(2,5)
>>> a.std(axis=0)
array([2.5 2.5 2.5 2.5 2.5])
>>> ak.std(a,axis=0)
array([2.5 2.5 2.5 2.5 2.5])
>>> a.std(axis=1)
array([1.4142135623730951 1.4142135623730951])
>>> ak.std(a,axis=1)
array([1.4142135623730951 1.4142135623730951])

Raises:

TypeError – Raised if pda is not a pdarray instance or ddof is not an integer
ValueError – Raised if ddof is an integer < 0
RuntimeError – Raised if there’s a server-side error thrown

See also

mean, var

Notes

The standard deviation is the square root of the average of the squared deviations from the mean, i.e., std = sqrt(mean((x - x.mean())**2)).

The average squared deviation is normally calculated as x.sum() / N, where N = len(x). If, however, ddof is specified, the divisor N - ddof is used instead. In standard statistical practice, ddof=1 provides an unbiased estimator of the variance of the infinite population. ddof=0 provides a maximum likelihood estimate of the variance for normally distributed variables. The standard deviation computed in this function is the square root of the estimated variance, so even with ddof=1, it will not be an unbiased estimate of the standard deviation per se.

sum(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numeric_scalars | pdarray[source]¶

Return sum of array elements along the given axis.

Parameters:

axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

numeric_scalars if axis is omitted, in which case operation is done over entire array pdarray if axis is supplied, in which case the operation is done along that axis

Return type:

numeric_scalars or pdarray

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

Examples

>>> import arkouda as ak
>>> ak.sum(ak.array([1,2,3,4,5]))
np.int64(15)
>>> ak.sum(ak.array([5.5,4.5,3.5,2.5,1.5]))
np.float64(17.5)
>>> ak.array([[1,2,3],[5,4,3]]).sum(axis=1)
array([6 12])

Notes

Works as a method of a pdarray (e.g. a.sum()) or a standalone function (e.g. ak.sum(a))

to_csv(prefix_path: str, dataset: str = 'array', col_delim: str = ',', overwrite: bool = False)[source]¶

Write pdarry to CSV file(s). File will contain a single column with the pdarray data. All CSV files written by Arkouda include a header denoting data types of the columns.

Parameters:

prefix_path (str) – filename prefix to be used for saving files. Files will have _LOCALE#### appended when they are written to disk.
dataset (str, defaults to "array") – column name to save the pdarray under.
col_delim (str, defaults to ",") – value to be used to separate columns within the file. Please be sure that the value used DOES NOT appear in your dataset.
overwrite (bool, defaults to False) – If True, existing files matching the provided path will be overwritten. if False and existing files are found, an error will be returned.

Returns:

response message

Return type:

str

Raises:

ValueError – Raised if all datasets are not present in all parquet files or if one or more of the specified files do not exist
RuntimeError – Raised if one or more of the specified files cannot be opened. if ‘allow_errors’ is true, this may be raised if no values are returned from the server.
TypeError – Raise if the server returns an unknown arkouda_type

Notes

CSV format is not currently supported by load/load_all operations
The column delimiter is expected to be the same for all column names and data
Be sure that column delimiters are not found within your data.
All CSV files must delimit rows using newline (”n”) at this time.

to_cuda()[source]¶

Convert the array to a Numba DeviceND array, transferring array data from the arkouda server to Python via ndarray. If the array exceeds a builtin size limit, a RuntimeError is raised.

Returns:

A Numba ndarray with the same attributes and data as the pdarray; on GPU

Return type:

numba.DeviceNDArray

Raises:

ImportError – Raised if CUDA is not available
ModuleNotFoundError – Raised if Numba is either not installed or not enabled
RuntimeError – Raised if there is a server-side error thrown in the course of retrieving the pdarray.

Notes

The number of bytes in the array cannot exceed client.maxTransferBytes, otherwise a RuntimeError will be raised. This is to protect the user from overflowing the memory of the system on which the Python client is running, under the assumption that the server is running on a distributed system with much more memory than the client. The user may override this limit by setting client.maxTransferBytes to a larger value, but proceed with caution.

See also

array

Examples

>>> import arkouda as ak
>>> a = ak.arange(0, 5, 1)
>>> a.to_cuda()
array([0, 1, 2, 3, 4])
>>> type(a.to_cuda())
numpy.devicendarray

to_hdf(prefix_path: str, dataset: str = 'array', mode: str = 'truncate', file_type: str = 'distribute') → str[source]¶

Save the pdarray to HDF5. The object can be saved to a collection of files or single file.

Parameters:

prefix_path (str) – Directory and filename prefix that all output files share
dataset (str) – Name of the dataset to create in files (must not already exist)
mode (str {'truncate' | 'append'}) – By default, truncate (overwrite) output files, if they exist. If ‘append’, attempt to create new dataset in existing files.
file_type (str ("single" | "distribute")) – Default: “distribute” When set to single, dataset is written to a single file. When distribute, dataset is written on a file per locale. This is only supported by HDF5 files and will have no impact of Parquet Files.

Return type:

string message indicating result of save operation

Raises:

RuntimeError – Raised if a server-side error is thrown saving the pdarray

Notes

The prefix_path must be visible to the arkouda server and the user must

have write permission. - Output files have names of the form <prefix_path>_LOCALE<i>, where <i> ranges from 0 to numLocales for file_type=’distribute’. Otherwise, the file name will be prefix_path. - If any of the output files already exist and the mode is ‘truncate’, they will be overwritten. If the mode is ‘append’ and the number of output files is less than the number of locales or a dataset with the same name already exists, a RuntimeError will result. - Any file extension can be used.The file I/O does not rely on the extension to determine the file format.

Examples

>>> import arkouda as ak
>>> a = ak.arange(25)

Saving without an extension >>> a.to_hdf(‘path/prefix’, dataset=’array’) # doctest: +SKIP Saves the array to numLocales HDF5 files with the name cwd/path/name_prefix_LOCALE####

Saving with an extension (HDF5) >>> a.to_hdf(‘path/prefix.h5’, dataset=’array’) # doctest: +SKIP Saves the array to numLocales HDF5 files with the name cwd/path/name_prefix_LOCALE####.h5 where #### is replaced by each locale number

Saving to a single file >>> a.to_hdf(‘path/prefix.hdf5’, dataset=’array’, file_type=’single’) # doctest: +SKIP Saves the array in to single hdf5 file on the root node. cwd/path/name_prefix.hdf5

to_list() → List[arkouda.numpy.dtypes.numeric_scalars][source]¶

Convert the array to a list, transferring array data from the Arkouda server to client-side Python. Note: if the pdarray size exceeds client.maxTransferBytes, a RuntimeError is raised.

Returns:: A list with the same data as the pdarray
Return type:: List[numeric_scalars]
Raises:: RuntimeError – Raised if there is a server-side error thrown, if the pdarray size exceeds the built-in client.maxTransferBytes size limit, or if the bytes received does not match expected number of bytes

Notes

See also

to_ndarray

Examples

>>> import arkouda as ak
>>> a = ak.arange(0, 5, 1)
>>> a.to_list()
[0, 1, 2, 3, 4]
>>> type(a.to_list())
<class 'list'>

to_ndarray() → numpy.ndarray[source]¶

Convert the array to a np.ndarray, transferring array data from the Arkouda server to client-side Python. Note: if the pdarray size exceeds client.maxTransferBytes, a RuntimeError is raised.

Returns:: A numpy ndarray with the same attributes and data as the pdarray
Return type:: np.ndarray
Raises:: RuntimeError – Raised if there is a server-side error thrown, if the pdarray size exceeds the built-in client.maxTransferBytes size limit, or if the bytes received does not match expected number of bytes

Notes

See also

array, to_list

Examples

>>> import arkouda as ak
>>> a = ak.arange(0, 5, 1)
>>> a.to_ndarray()
array([0, 1, 2, 3, 4])
>>> type(a.to_ndarray())
<class 'numpy.ndarray'>

to_parquet(prefix_path: str, dataset: str = 'array', mode: str = 'truncate', compression: str | None = None) → str[source]¶

Save the pdarray to Parquet. The result is a collection of files, one file per locale of the arkouda server, where each filename starts with prefix_path. Each locale saves its chunk of the array to its corresponding file.

Parameters:

prefix_path (str) – Directory and filename prefix that all output files share
dataset (str) – Name of the dataset to create in files (must not already exist)
mode (str {'truncate' | 'append'}) – By default, truncate (overwrite) output files, if they exist. If ‘append’, attempt to create new dataset in existing files.
compression (str (Optional)) – (None | “snappy” | “gzip” | “brotli” | “zstd” | “lz4”) Sets the compression type used with Parquet files

Return type:

string message indicating result of save operation

Raises:

RuntimeError – Raised if a server-side error is thrown saving the pdarray

Notes

The prefix_path must be visible to the arkouda server and the user must

have write permission. - Output files have names of the form <prefix_path>_LOCALE<i>, where <i> ranges from 0 to numLocales for file_type=’distribute’. - ‘append’ write mode is supported, but is not efficient. - If any of the output files already exist and the mode is ‘truncate’, they will be overwritten. If the mode is ‘append’ and the number of output files is less than the number of locales or a dataset with the same name already exists, a RuntimeError will result. - Any file extension can be used.The file I/O does not rely on the extension to determine the file format.

Examples

>>> import arkouda as ak
>>> a = ak.arange(25)

Saving without an extension >>> a.to_parquet(‘path/prefix’, dataset=’array’) # doctest: +SKIP Saves the array to numLocales HDF5 files with the name cwd/path/name_prefix_LOCALE####

Saving with an extension (HDF5) >>> a.to_parqet(‘path/prefix.parquet’, dataset=’array’) # doctest: +SKIP Saves the array to numLocales HDF5 files with the name cwd/path/name_prefix_LOCALE####.parquet where #### is replaced by each locale number

transfer(hostname: str, port: arkouda.numpy.dtypes.int_scalars)[source]¶

Send a pdarray to a different Arkouda server.

Parameters:

hostname (str) – The hostname where the Arkouda server intended to receive the pdarray is running.
port (int_scalars) – The port to send the array over. This needs to be an open port (i.e., not one that the Arkouda server is running on). This will open up numLocales ports, each of which in succession, so will use ports of the range {port..(port+numLocales)} (e.g., running an Arkouda server of 4 nodes, port 1234 is passed as port, Arkouda will use ports 1234, 1235, 1236, and 1237 to send the array data). This port much match the port passed to the call to ak.receive_array().

Return type:

A message indicating a complete transfer

Raises:

ValueError – Raised if the op is not within the pdarray.BinOps set
TypeError – Raised if other is not a pdarray or the pdarray.dtype is not a supported dtype

unregister() → None[source]¶

Unregister a pdarray in the arkouda server which was previously registered using register() and/or attahced to using attach()

Raises:: RuntimeError – Raised if the server could not find the internal name/symbol to remove

See also

register, unregister, is_registered, unregister_pdarray_by_name, list_registry

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> import arkouda as ak
>>> a = ak.zeros(3)
>>> a.register("my_zeros")
array([0.00000000000000000 0.00000000000000000 0.00000000000000000])

potentially disconnect from server and reconnect to server >>> b = ak.attach(“my_zeros”) >>> b.unregister()

update_hdf(prefix_path: str, dataset: str = 'array', repack: bool = True)[source]¶

Overwrite the dataset with the name provided with this pdarray. If the dataset does not exist it is added

Parameters:

prefix_path (str) – Directory and filename prefix that all output files share
dataset (str) – Name of the dataset to create in files
repack (bool) – Default: True HDF5 does not release memory on delete. When True, the inaccessible data (that was overwritten) is removed. When False, the data remains, but is inaccessible. Setting to false will yield better performance, but will cause file sizes to expand.

Return type:

str - success message if successful

Raises:

RuntimeError – Raised if a server-side error is thrown saving the pdarray

Notes

If file does not contain File_Format attribute to indicate how it was saved, the file name is checked for _LOCALE#### to determine if it is distributed.
If the dataset provided does not exist, it will be added

value_counts()[source]¶

Count the occurrences of the unique values of self.

Returns:

unique_valuespdarray: The unique values, sorted in ascending order
countspdarray, int64: The number of times the corresponding unique value occurs

Return type:

pdarray, pdarray|int64

Examples

>>> import arkouda as ak
>>> ak.array([2, 0, 2, 4, 0, 0]).value_counts()
(array([0 2 4]), array([3 2 1]))

var(ddof: arkouda.numpy.dtypes.int_scalars = 0, axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool | None = False) → numpy.float64 | pdarray[source]¶

Return the variance of values in the array.

Parameters:

ddof (int_scalars) – “Delta Degrees of Freedom” used in calculating var
axis (int, Tuple[int, ...], optional, default = None) – The axis or axes along which to do the operation If None, the computation is done across the entire array.
keepdims (bool, optional, default = False) – Whether to keep the singleton dimension(s) along axis in the result.

Returns:

The scalar variance of the array, or the variance along the axis/axes if supplied

Return type:

Union[np.float64, pdarray]

Examples

>>> import arkouda as ak
>>> a = ak.arange(10)
>>> ak.var(a)
np.float64(8.25)
>>> a.var()
np.float64(8.25)
>>> a = ak.arange(10).reshape(2,5)
>>> a.var(axis=0)
array([6.25 6.25 6.25 6.25 6.25])
>>> ak.var(a,axis=0)
array([6.25 6.25 6.25 6.25 6.25])
>>> a.var(axis=1)
array([2.00000000000000000 2.00000000000000000])
>>> ak.var(a,axis=1)
array([2.00000000000000000 2.00000000000000000])

Raises:

TypeError – Raised if pda is not a pdarray instance
ValueError – Raised if the ddof >= pdarray size
RuntimeError – Raised if there’s a server-side error thrown

See also

mean, std

Notes

The variance is the average of the squared deviations from the mean, i.e., var = mean((x - x.mean())**2).

The mean is normally calculated as x.sum() / N, where N = len(x). If, however, ddof is specified, the divisor N - ddof is used instead. In standard statistical practice, ddof=1 provides an unbiased estimator of the variance of a hypothetical infinite population. ddof=0 provides a maximum likelihood estimate of the variance for normally distributed variables.

class arkouda.matcher.str_scalars¶

Bases: _NotIterable

Mixin to prevent iteration, without being compatible with Iterable.

That is, we could do:

def __iter__(self): raise TypeError()

But this would make users of this mixin duck type-compatible with collections.abc.Iterable - isinstance(foo, Iterable) would be True.

Luckily, we can instead prevent iteration by setting __iter__ to None, which is treated specially.

copy_with(params)¶