arkouda.pdarrayclass¶

Exceptions¶

RegistrationError

Error/Exception used when the Arkouda Server cannot register an object

Classes¶

pdarray

The basic arkouda array class. This class contains only the

Functions¶

`argmaxk`(→ pdarray)	Find the indices corresponding to the k maximum values of an array.
`argmink`(→ pdarray)	Finds the indices corresponding to the k minimum values of an array.
`attach_pdarray`(→ pdarray)	class method to return a pdarray attached to the registered name in the arkouda
`broadcast_to_shape`(→ pdarray)	expand an array's rank to the specified shape using broadcasting
`clear`(→ None)	Send a clear message to clear all unregistered data from the server symbol table
`clz`(→ pdarray)	Count leading zeros for each integer in an array.
`corr`(→ numpy.float64)	Return the correlation between x and y
`cov`(→ numpy.float64)	Return the covariance of x and y
`ctz`(→ pdarray)	Count trailing zeros for each integer in an array.
`divmod`(→ Tuple[pdarray, pdarray])
`dot`(→ Union[arkouda.numpy.dtypes.numpy_scalars, pdarray])	Returns the sum of the elementwise product of two arrays of the same size (the dot product) or
`fmod`(→ pdarray)	Returns the element-wise remainder of division.
`maxk`(→ pdarray)	Find the k maximum values of an array.
`mean`(→ numpy.float64)	Return the mean of the array.
`mink`(→ pdarray)	Find the k minimum values of an array.
`mod`(→ pdarray)	Returns the element-wise remainder of division.
`parity`(→ pdarray)	Find the bit parity (XOR of all bits) for each integer in an array.
`popcount`(→ pdarray)	Find the population (number of bits set) for each integer in an array.
`power`(→ pdarray)	Raises an array to a power. If where is given, the operation will only take place in the positions
`rotl`(→ pdarray)	Rotate bits of <x> to the left by <rot>.
`rotr`(→ pdarray)	Rotate bits of <x> to the left by <rot>.
`sqrt`(→ pdarray)	Takes the square root of array. If where is given, the operation will only take place in
`std`(→ numpy.float64)	Return the standard deviation of values in the array. The standard
`unregister_pdarray_by_name`(→ None)	Unregister a named pdarray in the arkouda server which was previously
`var`(→ numpy.float64)	Return the variance of values in the array.

Module Contents¶

exception arkouda.pdarrayclass.RegistrationError[source]¶

Bases: Exception

Error/Exception used when the Arkouda Server cannot register an object

arkouda.pdarrayclass.argmaxk(pda: pdarray, k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Find the indices corresponding to the k maximum values of an array.

Returns the largest k values of an array, sorted

Parameters:

pda (pdarray) – Input array.
k (int_scalars) – The desired count of indices corresponding to maxmum array values

Returns:

The indices of the maximum k values from the pda, sorted

Return type:

pdarray, int

Raises:

TypeError – Raised if pda is not a pdarray or k is not an integer
ValueError – Raised if the pda is empty or k < 1

Notes

This call is equivalent in value to:

ak.argsort(a)[k:]

and generally outperforms this operation.

This reduction will see a significant drop in performance as k grows beyond a certain value. This value is system dependent, but generally about a k of 5 million is where performance degradation has been observed.

Examples

>>> A = ak.array([10,5,1,3,7,2,9,0])
>>> ak.argmaxk(A, 3)
array([4, 6, 0])
>>> ak.argmaxk(A, 4)
array([1, 4, 6, 0])

arkouda.pdarrayclass.argmink(pda: pdarray, k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Finds the indices corresponding to the k minimum values of an array.

Parameters:

pda (pdarray) – Input array.
k (int_scalars) – The desired count of indices corresponding to minimum array values

Returns:

The indices of the minimum k values from the pda, sorted

Return type:

pdarray, int

Raises:

TypeError – Raised if pda is not a pdarray or k is not an integer
ValueError – Raised if the pda is empty or k < 1

Notes

This call is equivalent in value to:

ak.argsort(a)[:k]

and generally outperforms this operation.

Examples

>>> A = ak.array([10,5,1,3,7,2,9,0])
>>> ak.argmink(A, 3)
array([7, 2, 5])
>>> ak.argmink(A, 4)
array([7, 2, 5, 3])

arkouda.pdarrayclass.attach_pdarray(user_defined_name: str) → pdarray[source]¶

class method to return a pdarray attached to the registered name in the arkouda server which was registered using register()

Parameters:: user_defined_name (str) – user defined name which array was registered under
Returns:: pdarray which is bound to the corresponding server side component which was registered with user_defined_name
Return type:: pdarray
Raises:: TypeError – Raised if user_defined_name is not a str

See also

attach, register, unregister, is_registered, unregister_pdarray_by_name, list_registry

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> a = zeros(100)
>>> a.register("my_zeros")
>>> # potentially disconnect from server and reconnect to server
>>> b = ak.attach_pdarray("my_zeros")
>>> # ...other work...
>>> b.unregister()

arkouda.pdarrayclass.broadcast_to_shape(pda: pdarray, shape: Tuple[int, Ellipsis]) → pdarray[source]¶: expand an array’s rank to the specified shape using broadcasting

arkouda.pdarrayclass.clear() → None[source]¶

Send a clear message to clear all unregistered data from the server symbol table

Return type:: None
Raises:: RuntimeError – Raised if there is a server-side error in executing clear request

arkouda.pdarrayclass.clz(pda: pdarray) → pdarray[source]¶

Count leading zeros for each integer in an array.

Parameters:: pda (pdarray, int64, uint64, bigint) – Input array (must be integral).
Returns:: lz – The number of leading zeros of each element.
Return type:: pdarray
Raises:: TypeError – If input array is not int64, uint64, or bigint

Examples

>>> A = ak.arange(10)
>>> ak.clz(A)
array([64, 63, 62, 62, 61, 61, 61, 61, 60, 60])

arkouda.pdarrayclass.corr(x: pdarray, y: pdarray) → numpy.float64[source]¶

Return the correlation between x and y

Parameters:

x (pdarray) – One of the pdarrays used to calculate correlation
y (pdarray) – One of the pdarrays used to calculate correlation

Returns:

The scalar correlation of the two pdarrays

Return type:

np.float64

Raises:

TypeError – Raised if x or y is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

See also

std, cov

Notes

The correlation is calculated by cov(x, y) / (x.std(ddof=1) * y.std(ddof=1))

arkouda.pdarrayclass.cov(x: pdarray, y: pdarray) → numpy.float64[source]¶

Return the covariance of x and y

Parameters:

x (pdarray) – One of the pdarrays used to calculate covariance
y (pdarray) – One of the pdarrays used to calculate covariance

Returns:

The scalar covariance of the two pdarrays

Return type:

np.float64

Raises:

TypeError – Raised if x or y is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

See also

mean, var

Notes

The covariance is calculated by cov = ((x - x.mean()) * (y - y.mean())).sum() / (x.size - 1).

arkouda.pdarrayclass.ctz(pda: pdarray) → pdarray[source]¶

Count trailing zeros for each integer in an array.

Parameters:: pda (pdarray, int64, uint64, bigint) – Input array (must be integral).
Returns:: lz – The number of trailing zeros of each element.
Return type:: pdarray

Notes

ctz(0) is defined to be zero.

Raises:: TypeError – If input array is not int64, uint64, or bigint

Examples

>>> A = ak.arange(10)
>>> ak.ctz(A)
array([0, 0, 1, 0, 2, 0, 1, 0, 3, 0])

arkouda.pdarrayclass.divmod(x: arkouda.numpy.dtypes.numeric_scalars | pdarray, y: arkouda.numpy.dtypes.numeric_scalars | pdarray, where: arkouda.numpy.dtypes.bool_scalars | pdarray = True) → Tuple[pdarray, pdarray][source]¶

Parameters:

x (numeric_scalars(float_scalars, int_scalars) or pdarray) – The dividend array, the values that will be the numerator of the floordivision and will be acted on by the bases for modular division.
y (numeric_scalars(float_scalars, int_scalars) or pdarray) – The divisor array, the values that will be the denominator of the division and will be the bases for the modular division.
where (Boolean or pdarray) – This condition is broadcast over the input. At locations where the condition is True, the corresponding value will be divided using floor and modular division. Elsewhere, it will retain its original value. Default set to True.

Returns:

Returns a tuple that contains quotient and remainder of the division

Return type:

(pdarray, pdarray)

Raises:

TypeError – At least one entry must be a pdarray
ValueError – If both inputs are both pdarrays, their size must match
ZeroDivisionError – No entry in y is allowed to be 0, to prevent division by zero

Notes

The div is calculated by x // y The mod is calculated by x % y

Examples

>>> x = ak.arange(5, 10)
>>> y = ak.array([2, 1, 4, 5, 8])
>>> ak.divmod(x,y)
(array([2 6 1 1 1]), array([1 0 3 3 1]))
>>> ak.divmod(x,y, x % 2 == 0)
(array([5 6 7 1 9]), array([5 0 7 3 9]))

Returns the sum of the elementwise product of two arrays of the same size (the dot product) or the product of a singleton element and an array.

Parameters:

pda1 (Union[numeric_scalars, pdarray])
pda2 (Union[numeric_scalars, pdarray])

Returns:

The sum of the elementwise product pda1 and pda2 or the product of a singleton element and an array.

Return type:

Union[numeric_scalars, pdarray]

Raises:

ValueError – Raised if the size of pda1 is not the same as pda2

Examples

>>> x = ak.array([2, 3])
>>> y = ak.array([4, 5])
>>> ak.dot(x,y)
23
>>> ak.dot(x,2)
array([4 6])

arkouda.pdarrayclass.fmod(dividend: pdarray | arkouda.numpy.dtypes.numeric_scalars, divisor: pdarray | arkouda.numpy.dtypes.numeric_scalars) → pdarray[source]¶

Returns the element-wise remainder of division.

It is equivalent to np.fmod, the remainder has the same sign as the dividend.

Parameters:

dividend (numeric scalars or pdarray) – The array being acted on by the bases for the modular division.
divisor (numeric scalars or pdarray) – The array that will be the bases for the modular division.

Returns:

Returns an array that contains the element-wise remainder of division.

Return type:

pdarray

arkouda.pdarrayclass.maxk(pda: pdarray, k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Find the k maximum values of an array.

Returns the largest k values of an array, sorted

Parameters:

pda (pdarray) – Input array.
k (int_scalars) – The desired count of maximum values to be returned by the output.

Returns:

The maximum k values from pda, sorted

Return type:

pdarray, int

Raises:

TypeError – Raised if pda is not a pdarray or k is not an integer
ValueError – Raised if the pda is empty or k < 1

Notes

This call is equivalent in value to:

a[ak.argsort(a)[k:]]

and generally outperforms this operation.

Examples

>>> A = ak.array([10,5,1,3,7,2,9,0])
>>> ak.maxk(A, 3)
array([7, 9, 10])
>>> ak.maxk(A, 4)
array([5, 7, 9, 10])

arkouda.pdarrayclass.mean(pda: pdarray) → numpy.float64[source]¶

Return the mean of the array.

Parameters:

pda (pdarray) – Values for which to calculate the mean

Returns:

The mean calculated from the pda sum and size

Return type:

np.float64

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

arkouda.pdarrayclass.mink(pda: pdarray, k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Find the k minimum values of an array.

Returns the smallest k values of an array, sorted

Parameters:

pda (pdarray) – Input array.
k (int_scalars) – The desired count of minimum values to be returned by the output.

Returns:

The minimum k values from pda, sorted

Return type:

pdarray

Raises:

TypeError – Raised if pda is not a pdarray
ValueError – Raised if the pda is empty or k < 1

Notes

This call is equivalent in value to:

a[ak.argsort(a)[:k]]

and generally outperforms this operation.

Examples

>>> A = ak.array([10,5,1,3,7,2,9,0])
>>> ak.mink(A, 3)
array([0, 1, 2])
>>> ak.mink(A, 4)
array([0, 1, 2, 3])

arkouda.pdarrayclass.mod(dividend, divisor) → pdarray[source]¶

Returns the element-wise remainder of division.

Computes the remainder complementary to the floor_divide function. It is equivalent to np.mod, the remainder has the same sign as the divisor.

Parameters:

dividend – The array being acted on by the bases for the modular division.
divisor – The array that will be the bases for the modular division.

Returns:

Returns an array that contains the element-wise remainder of division.

Return type:

pdarray

arkouda.pdarrayclass.parity(pda: pdarray) → pdarray[source]¶

Find the bit parity (XOR of all bits) for each integer in an array.

Parameters:: pda (pdarray, int64, uint64, bigint) – Input array (must be integral).
Returns:: parity – The parity of each element: 0 if even number of bits set, 1 if odd.
Return type:: pdarray
Raises:: TypeError – If input array is not int64, uint64, or bigint

Examples

>>> A = ak.arange(10)
>>> ak.parity(A)
array([0, 1, 1, 0, 1, 0, 0, 1, 1, 0])

class arkouda.pdarrayclass.pdarray(name: str, mydtype: numpy.dtype | str, size: arkouda.numpy.dtypes.int_scalars, ndim: arkouda.numpy.dtypes.int_scalars, shape: Sequence[int], itemsize: arkouda.numpy.dtypes.int_scalars, max_bits: int | None = None)[source]¶

The basic arkouda array class. This class contains only the attributies of the array; the data resides on the arkouda server. When a server operation results in a new array, arkouda will create a pdarray instance that points to the array data on the server. As such, the user should not initialize pdarray instances directly.

name¶

The server-side identifier for the array

Type:: str

dtype¶

The element type of the array

Type:: dtype

size¶

The number of elements in the array

Type:: int_scalars

ndim¶

The rank of the array (currently only rank 1 arrays supported)

Type:: int_scalars

shape¶

A list or tuple containing the sizes of each dimension of the array

Type:: Sequence[int]

itemsize¶

The size in bytes of each element

Type:: int_scalars

BinOps¶

OpEqOps¶

all(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.bool_scalars | pdarray[source]¶: Return True iff all elements of the array evaluate to True.

any(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.bool_scalars | pdarray[source]¶: Return True iff any element of the array evaluates to True.

argmax(axis: int | None | None = None, keepdims: bool = False) → numpy.int64 | numpy.uint64 | pdarray[source]¶: Return the index of the first occurrence of the array max value.

argmaxk(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Finds the indices corresponding to the maximum “k” values.

Parameters:: k (int_scalars) – The desired count of maximum values to be returned by the output.
Returns:: Indices corresponding to the maximum k values, sorted
Return type:: pdarray, int
Raises:: TypeError – Raised if pda is not a pdarray

argmin(axis: int | None | None = None, keepdims: bool = False) → numpy.int64 | numpy.uint64 | pdarray[source]¶: Return the index of the first occurrence of the array min value

argmink(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Compute the minimum “k” values.

Parameters:: k (int_scalars) – The desired count of maximum values to be returned by the output.
Returns:: Indices corresponding to the maximum k values from pda
Return type:: pdarray, int
Raises:: TypeError – Raised if pda is not a pdarray

astype(dtype) → pdarray[source]¶

Cast values of pdarray to provided dtype

Parameters:: dtype (np.dtype or str) – Dtype to cast to
Returns:: An arkouda pdarray with values converted to the specified data type
Return type:: ak.pdarray

Notes

This is essentially shorthand for ak.cast(x, ‘<dtype>’) where x is a pdarray.

static attach(user_defined_name: str) → pdarray[source]¶

class method to return a pdarray attached to the registered name in the arkouda server which was registered using register()

Parameters:: user_defined_name (str) – user defined name which array was registered under
Returns:: pdarray which is bound to the corresponding server side component which was registered with user_defined_name
Return type:: pdarray
Raises:: TypeError – Raised if user_defined_name is not a str

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> a = zeros(100)
>>> a.register("my_zeros")
>>> # potentially disconnect from server and reconnect to server
>>> b = ak.pdarray.attach("my_zeros")
>>> # ...other work...
>>> b.unregister()

bigint_to_uint_arrays() → List[pdarray][source]¶

Creates a list of uint pdarrays from a bigint pdarray. The first item in return will be the highest 64 bits of the bigint pdarray and the last item will be the lowest 64 bits.

Returns:: A list of uint pdarrays where: The first item in return will be the highest 64 bits of the bigint pdarray and the last item will be the lowest 64 bits.
Return type:: List[pdarrays]
Raises:: RuntimeError – Raised if there is a server-side error thrown

See also

pdarraycreation.bigint_from_uint_arrays

Examples

>>> a = ak.arange(2**64, 2**64 + 5)
>>> a
array(["18446744073709551616" "18446744073709551617" "18446744073709551618"
"18446744073709551619" "18446744073709551620"])

>>> a.bigint_to_uint_arrays()
[array([1 1 1 1 1]), array([0 1 2 3 4])]

clz() → pdarray[source]¶: Count the number of leading zeros in each element. See ak.clz.

corr(y: pdarray) → numpy.float64[source]¶

Compute the correlation between self and y using pearson correlation coefficient.

Parameters:

y (pdarray) – Other pdarray used to calculate correlation

Returns:

The scalar correlation of the two arrays

Return type:

np.float64

Raises:

TypeError – Raised if y is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

cov(y: pdarray) → numpy.float64[source]¶

Compute the covariance between self and y.

Parameters:

y (pdarray) – Other pdarray used to calculate covariance

Returns:

The scalar covariance of the two arrays

Return type:

np.float64

Raises:

TypeError – Raised if y is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

ctz() → pdarray[source]¶: Count the number of trailing zeros in each element. See ak.ctz.

dtype¶

equals(other) → arkouda.numpy.dtypes.bool_scalars[source]¶

Whether pdarrays are the same size and all entries are equal.

Parameters:: other (object) – object to compare.
Returns:: True if the pdarrays are the same, o.w. False.
Return type:: bool

Examples

>>> import arkouda as ak
>>> ak.connect()
>>> a = ak.array([1, 2, 3])
>>> a_cpy = ak.array([1, 2, 3])
>>> a.equals(a_cpy)
True
>>> a2 = ak.array([1, 2, 5)
>>> a.equals(a2)
False

fill(value: arkouda.numpy.dtypes.numeric_scalars) → None[source]¶

Fill the array (in place) with a constant value.

Parameters:: value (numeric_scalars)
Raises:: TypeError – Raised if value is not an int, int64, float, or float64

flatten()[source]¶

Return a copy of the array collapsed into one dimension.

Return type:: A copy of the input array, flattened to one dimension.

format_other(other) → str[source]¶

Attempt to cast scalar other to the element dtype of this pdarray, and print the resulting value to a string (e.g. for sending to a server command). The user should not call this function directly.

Parameters:: other (object) – The scalar to be cast to the pdarray.dtype
Return type:: string representation of np.dtype corresponding to the other parameter
Raises:: TypeError – Raised if the other parameter cannot be converted to Numpy dtype

property inferred_type: str | None¶: Return a string of the type inferred from the values.

info() → str[source]¶

Returns a JSON formatted string containing information about all components of self

Parameters:: None
Returns:: JSON string containing information about all components of self
Return type:: str

is_registered() → numpy.bool_[source]¶

Return True iff the object is contained in the registry

Parameters:: None
Returns:: Indicates if the object is contained in the registry
Return type:: bool
Raises:: RuntimeError – Raised if there’s a server-side error thrown

Note

This will return True if the object is registered itself or as a component of another object

is_sorted(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.bool_scalars | pdarray[source]¶

Return True iff the array is monotonically non-decreasing.

Parameters:

None

Returns:

Indicates if the array is monotonically non-decreasing

Return type:

bool

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

itemsize¶

max(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numpy_scalars | pdarray[source]¶: Return the maximum value of the array.

property max_bits¶

maxk(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Compute the maximum “k” values.

Parameters:: k (int_scalars) – The desired count of maximum values to be returned by the output.
Returns:: The maximum k values from pda
Return type:: pdarray, int
Raises:: TypeError – Raised if pda is not a pdarray

mean() → numpy.float64[source]¶: Return the mean of the array.

min(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numpy_scalars | pdarray[source]¶: Return the minimum value of the array.

mink(k: arkouda.numpy.dtypes.int_scalars) → pdarray[source]¶

Compute the minimum “k” values.

Parameters:: k (int_scalars) – The desired count of maximum values to be returned by the output.
Returns:: The maximum k values from pda
Return type:: pdarray, int
Raises:: TypeError – Raised if pda is not a pdarray

name¶

property nbytes¶

The size of the pdarray in bytes.

Returns:: The size of the pdarray in bytes.
Return type:: int

ndim¶

objType = 'pdarray'¶

opeq(other, op)[source]¶

parity() → pdarray[source]¶: Find the parity (XOR of all bits) in each element. See ak.parity.

popcount() → pdarray[source]¶: Find the population (number of bits set) in each element. See ak.popcount.

pretty_print_info() → None[source]¶

Prints information about all components of self in a human readable format

Parameters:: None
Return type:: None

prod(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numpy_scalars | pdarray[source]¶: Return the product of all elements in the array. Return value is always a np.float64 or np.int64.

register(user_defined_name: str) → pdarray[source]¶

Register this pdarray with a user defined name in the arkouda server so it can be attached to later using pdarray.attach() This is an in-place operation, registering a pdarray more than once will update the name in the registry and remove the previously registered name. A name can only be registered to one pdarray at a time.

Parameters:

user_defined_name (str) – user defined name array is to be registered under

Returns:

The same pdarray which is now registered with the arkouda server and has an updated name. This is an in-place modification, the original is returned to support a fluid programming style. Please note you cannot register two different pdarrays with the same name.

Return type:

pdarray

Raises:

TypeError – Raised if user_defined_name is not a str
RegistrationError – If the server was unable to register the pdarray with the user_defined_name If the user is attempting to register more than one pdarray with the same name, the former should be unregistered first to free up the registration name.

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> a = zeros(100)
>>> a.register("my_zeros")
>>> # potentially disconnect from server and reconnect to server
>>> b = ak.pdarray.attach("my_zeros")
>>> # ...other work...
>>> b.unregister()

registered_name: str | None = None¶

reshape(*shape)[source]¶

Gives a new shape to an array without changing its data.

Parameters:: shape (int, tuple of ints, or pdarray) – The new shape should be compatible with the original shape.
Returns:: a pdarray with the same data, reshaped to the new shape
Return type:: pdarray

rotl(other) → pdarray[source]¶: Rotate bits left by <other>.

rotr(other) → pdarray[source]¶: Rotate bits right by <other>.

save(prefix_path: str, dataset: str = 'array', mode: str = 'truncate', compression: str | None = None, file_format: str = 'HDF5', file_type: str = 'distribute') → str[source]¶

DEPRECATED Save the pdarray to HDF5 or Parquet. The result is a collection of files, one file per locale of the arkouda server, where each filename starts with prefix_path. HDF5 support single files, in which case the file name will only be that provided. Each locale saves its chunk of the array to its corresponding file. :param prefix_path: Directory and filename prefix that all output files share :type prefix_path: str :param dataset: Name of the dataset to create in files (must not already exist) :type dataset: str :param mode: By default, truncate (overwrite) output files, if they exist.

If ‘append’, attempt to create new dataset in existing files.

Parameters:

compression (str (Optional)) – (None | “snappy” | “gzip” | “brotli” | “zstd” | “lz4”) Sets the compression type used with Parquet files
file_format (str {'HDF5', 'Parquet'}) – By default, saved files will be written to the HDF5 file format. If ‘Parquet’, the files will be written to the Parquet file format. This is case insensitive.
file_type (str ("single" | "distribute")) – Default: “distribute” When set to single, dataset is written to a single file. When distribute, dataset is written on a file per locale. This is only supported by HDF5 files and will have no impact of Parquet Files.

Return type:

string message indicating result of save operation

Raises:

RuntimeError – Raised if a server-side error is thrown saving the pdarray
ValueError – Raised if there is an error in parsing the prefix path pointing to file write location or if the mode parameter is neither truncate nor append
TypeError – Raised if any one of the prefix_path, dataset, or mode parameters is not a string

See also

save_all, load, read, to_parquet, to_hdf

Notes

The prefix_path must be visible to the arkouda server and the user must have write permission. Output files have names of the form <prefix_path>_LOCALE, where  ranges from 0 to numLocales. If any of the output files already exist and the mode is ‘truncate’, they will be overwritten. If the mode is ‘append’ and the number of output files is less than the number of locales or a dataset with the same name already exists, a RuntimeError will result. Previously all files saved in Parquet format were saved with a .parquet file extension. This will require you to use load as if you saved the file with the extension. Try this if an older file is not being found. Any file extension can be used.The file I/O does not rely on the extension to determine the file format.

Examples

>>> a = ak.arange(25)
>>> # Saving without an extension
>>> a.save('path/prefix', dataset='array')
Saves the array to numLocales HDF5 files with the name ``cwd/path/name_prefix_LOCALE####``
>>> # Saving with an extension (HDF5)
>>> a.save('path/prefix.h5', dataset='array')
Saves the array to numLocales HDF5 files with the name
``cwd/path/name_prefix_LOCALE####.h5`` where #### is replaced by each locale number
>>> # Saving with an extension (Parquet)
>>> a.save('path/prefix.parquet', dataset='array', file_format='Parquet')
Saves the array in numLocales Parquet files with the name
``cwd/path/name_prefix_LOCALE####.parquet`` where #### is replaced by each locale number

property shape¶

Return the shape of an array.

Returns:: The elements of the shape tuple give the lengths of the corresponding array dimensions.
Return type:: tuple of int

size¶

slice_bits(low, high) → pdarray[source]¶

Returns a pdarray containing only bits from low to high of self.

This is zero indexed and inclusive on both ends, so slicing the bottom 64 bits is pda.slice_bits(0, 63)

Parameters:

low (int) – The lowest bit included in the slice (inclusive) zero indexed, so the first bit is 0
high (int) – The highest bit included in the slice (inclusive)

Returns:

A new pdarray containing the bits of self from low to high

Return type:

pdarray

Raises:

RuntimeError – Raised if there is a server-side error thrown

Examples

>>> p = ak.array([2**65 + (2**64 - 1)])
>>> bin(p[0])
'0b101111111111111111111111111111111111111111111111111111111111111111'

>>> bin(p.slice_bits(64, 65)[0])
'0b10'

std(ddof: arkouda.numpy.dtypes.int_scalars = 0) → numpy.float64[source]¶

Compute the standard deviation. See arkouda.std for details.

Parameters:

ddof (int_scalars) – “Delta Degrees of Freedom” used in calculating std

Returns:

The scalar standard deviation of the array

Return type:

np.float64

Raises:

TypeError – Raised if pda is not a pdarray instance
RuntimeError – Raised if there’s a server-side error thrown

sum(axis: int | Tuple[int, Ellipsis] | None = None, keepdims: bool = False) → arkouda.numpy.dtypes.numpy_scalars | pdarray[source]¶: Return the sum of all elements in the array.

to_csv(prefix_path: str, dataset: str = 'array', col_delim: str = ',', overwrite: bool = False)[source]¶

Write pdarray to CSV file(s). File will contain a single column with the pdarray data. All CSV Files written by Arkouda include a header denoting data types of the columns.

prefix_path: str
The filename prefix to be used for saving files. Files will have _LOCALE#### appended when they are written to disk.

dataset: str
Column name to save the pdarray under. Defaults to “array”.

col_delim: str
Defaults to “,”. Value to be used to separate columns within the file. Please be sure that the value used DOES NOT appear in your dataset.

overwrite: bool
Defaults to False. If True, any existing files matching your provided prefix_path will be overwritten. If False, an error will be returned if existing files are found.

str reponse message

ValueError
Raised if all datasets are not present in all parquet files or if one or more of the specified files do not exist

RuntimeError
Raised if one or more of the specified files cannot be opened. If allow_errors is true this may be raised if no values are returned from the server.

TypeError
Raised if we receive an unknown arkouda_type returned from the server

CSV format is not currently supported by load/load_all operations

The column delimiter is expected to be the same for column names and data

Be sure that column delimiters are not found within your data.

All CSV files must delimit rows using newline (`

`) at this time.

to_cuda()[source]¶

Convert the array to a Numba DeviceND array, transferring array data from the arkouda server to Python via ndarray. If the array exceeds a builtin size limit, a RuntimeError is raised.

Returns:

A Numba ndarray with the same attributes and data as the pdarray; on GPU

Return type:

numba.DeviceNDArray

Raises:

ImportError – Raised if CUDA is not available
ModuleNotFoundError – Raised if Numba is either not installed or not enabled
RuntimeError – Raised if there is a server-side error thrown in the course of retrieving the pdarray.

Notes

The number of bytes in the array cannot exceed client.maxTransferBytes, otherwise a RuntimeError will be raised. This is to protect the user from overflowing the memory of the system on which the Python client is running, under the assumption that the server is running on a distributed system with much more memory than the client. The user may override this limit by setting client.maxTransferBytes to a larger value, but proceed with caution.

See also

array

Examples

>>> a = ak.arange(0, 5, 1)
>>> a.to_cuda()
array([0, 1, 2, 3, 4])

>>> type(a.to_cuda())
numpy.devicendarray

to_hdf(prefix_path: str, dataset: str = 'array', mode: str = 'truncate', file_type: str = 'distribute') → str[source]¶

Save the pdarray to HDF5. The object can be saved to a collection of files or single file. :param prefix_path: Directory and filename prefix that all output files share :type prefix_path: str :param dataset: Name of the dataset to create in files (must not already exist) :type dataset: str :param mode: By default, truncate (overwrite) output files, if they exist.

If ‘append’, attempt to create new dataset in existing files.

Parameters:: file_type (str ("single" | "distribute")) – Default: “distribute” When set to single, dataset is written to a single file. When distribute, dataset is written on a file per locale. This is only supported by HDF5 files and will have no impact of Parquet Files.
Return type:: string message indicating result of save operation
Raises:: RuntimeError – Raised if a server-side error is thrown saving the pdarray

Notes

The prefix_path must be visible to the arkouda server and the user must

have write permission. - Output files have names of the form <prefix_path>_LOCALE, where  ranges from 0 to numLocales for file_type=’distribute’. Otherwise, the file name will be prefix_path. - If any of the output files already exist and the mode is ‘truncate’, they will be overwritten. If the mode is ‘append’ and the number of output files is less than the number of locales or a dataset with the same name already exists, a RuntimeError will result. - Any file extension can be used.The file I/O does not rely on the extension to determine the file format.

Examples

>>> a = ak.arange(25)
>>> # Saving without an extension
>>> a.to_hdf('path/prefix', dataset='array')
Saves the array to numLocales HDF5 files with the name ``cwd/path/name_prefix_LOCALE####``
>>> # Saving with an extension (HDF5)
>>> a.to_hdf('path/prefix.h5', dataset='array')
Saves the array to numLocales HDF5 files with the name
``cwd/path/name_prefix_LOCALE####.h5`` where #### is replaced by each locale number
>>> # Saving to a single file
>>> a.to_hdf('path/prefix.hdf5', dataset='array', file_type='single')
Saves the array in to single hdf5 file on the root node.
``cwd/path/name_prefix.hdf5``

to_list() → List[source]¶

Convert the array to a list, transferring array data from the Arkouda server to client-side Python. Note: if the pdarray size exceeds client.maxTransferBytes, a RuntimeError is raised.

Returns:: A list with the same data as the pdarray
Return type:: list
Raises:: RuntimeError – Raised if there is a server-side error thrown, if the pdarray size exceeds the built-in client.maxTransferBytes size limit, or if the bytes received does not match expected number of bytes

Notes

See also

to_ndarray

Examples

>>> a = ak.arange(0, 5, 1)
>>> a.to_list()
[0, 1, 2, 3, 4]

>>> type(a.to_list())
list

to_ndarray() → numpy.ndarray[source]¶

Convert the array to a np.ndarray, transferring array data from the Arkouda server to client-side Python. Note: if the pdarray size exceeds client.maxTransferBytes, a RuntimeError is raised.

Returns:: A numpy ndarray with the same attributes and data as the pdarray
Return type:: np.ndarray
Raises:: RuntimeError – Raised if there is a server-side error thrown, if the pdarray size exceeds the built-in client.maxTransferBytes size limit, or if the bytes received does not match expected number of bytes

Notes

See also

array, to_list

Examples

>>> a = ak.arange(0, 5, 1)
>>> a.to_ndarray()
array([0, 1, 2, 3, 4])

>>> type(a.to_ndarray())
numpy.ndarray

to_parquet(prefix_path: str, dataset: str = 'array', mode: str = 'truncate', compression: str | None = None) → str[source]¶

Save the pdarray to Parquet. The result is a collection of files, one file per locale of the arkouda server, where each filename starts with prefix_path. Each locale saves its chunk of the array to its corresponding file. :param prefix_path: Directory and filename prefix that all output files share :type prefix_path: str :param dataset: Name of the dataset to create in files (must not already exist) :type dataset: str :param mode: By default, truncate (overwrite) output files, if they exist.

If ‘append’, attempt to create new dataset in existing files.

Parameters:: compression (str (Optional)) – (None | “snappy” | “gzip” | “brotli” | “zstd” | “lz4”) Sets the compression type used with Parquet files
Return type:: string message indicating result of save operation
Raises:: RuntimeError – Raised if a server-side error is thrown saving the pdarray

Notes

The prefix_path must be visible to the arkouda server and the user must

have write permission. - Output files have names of the form <prefix_path>_LOCALE, where  ranges from 0 to numLocales for file_type=’distribute’. - ‘append’ write mode is supported, but is not efficient. - If any of the output files already exist and the mode is ‘truncate’, they will be overwritten. If the mode is ‘append’ and the number of output files is less than the number of locales or a dataset with the same name already exists, a RuntimeError will result. - Any file extension can be used.The file I/O does not rely on the extension to determine the file format.

Examples

>>> a = ak.arange(25)
>>> # Saving without an extension
>>> a.to_parquet('path/prefix', dataset='array')
Saves the array to numLocales HDF5 files with the name ``cwd/path/name_prefix_LOCALE####``
>>> # Saving with an extension (HDF5)
>>> a.to_parqet('path/prefix.parquet', dataset='array')
Saves the array to numLocales HDF5 files with the name
``cwd/path/name_prefix_LOCALE####.parquet`` where #### is replaced by each locale number

transfer(hostname: str, port: arkouda.numpy.dtypes.int_scalars)[source]¶

Sends a pdarray to a different Arkouda server

Parameters:

hostname (str) – The hostname where the Arkouda server intended to receive the pdarray is running.
port (int_scalars) – The port to send the array over. This needs to be an open port (i.e., not one that the Arkouda server is running on). This will open up numLocales ports, each of which in succession, so will use ports of the range {port..(port+numLocales)} (e.g., running an Arkouda server of 4 nodes, port 1234 is passed as port, Arkouda will use ports 1234, 1235, 1236, and 1237 to send the array data). This port much match the port passed to the call to ak.receive_array().

Return type:

A message indicating a complete transfer

Raises:

ValueError – Raised if the op is not within the pdarray.BinOps set
TypeError – Raised if other is not a pdarray or the pdarray.dtype is not a supported dtype

unregister() → None[source]¶

Unregister a pdarray in the arkouda server which was previously registered using register() and/or attahced to using attach()

Return type:: None
Raises:: RuntimeError – Raised if the server could not find the internal name/symbol to remove

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> a = zeros(100)
>>> a.register("my_zeros")
>>> # potentially disconnect from server and reconnect to server
>>> b = ak.pdarray.attach("my_zeros")
>>> # ...other work...
>>> b.unregister()

update_hdf(prefix_path: str, dataset: str = 'array', repack: bool = True)[source]¶

Overwrite the dataset with the name provided with this pdarray. If the dataset does not exist it is added

Parameters:

prefix_path (str) – Directory and filename prefix that all output files share
dataset (str) – Name of the dataset to create in files
repack (bool) – Default: True HDF5 does not release memory on delete. When True, the inaccessible data (that was overwritten) is removed. When False, the data remains, but is inaccessible. Setting to false will yield better performance, but will cause file sizes to expand.

Return type:

str - success message if successful

Raises:

RuntimeError – Raised if a server-side error is thrown saving the pdarray

Notes

If file does not contain File_Format attribute to indicate how it was saved, the file name is checked for _LOCALE#### to determine if it is distributed.
If the dataset provided does not exist, it will be added

value_counts()[source]¶

Count the occurrences of the unique values of self.

Returns:

unique_values (pdarray) – The unique values, sorted in ascending order
counts (pdarray, int64) – The number of times the corresponding unique value occurs

Examples

>>> ak.array([2, 0, 2, 4, 0, 0]).value_counts()
(array([0, 2, 4]), array([3, 2, 1]))

var(ddof: arkouda.numpy.dtypes.int_scalars = 0) → numpy.float64[source]¶

Compute the variance. See arkouda.var for details.

Parameters:

ddof (int_scalars) – “Delta Degrees of Freedom” used in calculating var

Returns:

The scalar variance of the array

Return type:

np.float64

Raises:

TypeError – Raised if pda is not a pdarray instance
ValueError – Raised if the ddof >= pdarray size
RuntimeError – Raised if there’s a server-side error thrown

arkouda.pdarrayclass.popcount(pda: pdarray) → pdarray[source]¶

Find the population (number of bits set) for each integer in an array.

Parameters:: pda (pdarray, int64, uint64, bigint) – Input array (must be integral).
Returns:: population – The number of bits set (1) in each element
Return type:: pdarray
Raises:: TypeError – If input array is not int64, uint64, or bigint

Examples

>>> A = ak.arange(10)
>>> ak.popcount(A)
array([0, 1, 1, 2, 1, 2, 2, 3, 1, 2])

arkouda.pdarrayclass.power(pda: pdarray, pwr: int | float | pdarray, where: arkouda.numpy.dtypes.bool_scalars | pdarray = True) → pdarray[source]¶

Raises an array to a power. If where is given, the operation will only take place in the positions where the where condition is True.

Note: Our implementation of the where argument deviates from numpy. The difference in behavior occurs at positions where the where argument contains a False. In numpy, these position will have uninitialized memory (which can contain anything and will vary between runs). We have chosen to instead return the value of the original array in these positions.

Parameters:

pda (pdarray) – A pdarray of values that will be raised to a power (pwr)
pwr (integer, float, or pdarray) – The power(s) that pda is raised to
where (Boolean or pdarray) – This condition is broadcast over the input. At locations where the condition is True, the corresponding value will be raised to the respective power. Elsewhere, it will retain its original value. Default set to True.

Returns:

pdarray Returns a pdarray of values raised to a power, under the boolean where condition.

Examples

>>> a = ak.arange(5)
>>> ak.power(a, 3)
array([0, 1, 8, 27, 64])
>>> ak.power(a), 3, a % 2 == 0)
array([0, 1, 8, 3, 64])

arkouda.pdarrayclass.rotl(x, rot) → pdarray[source]¶

Rotate bits of <x> to the left by <rot>.

Parameters:

x (pdarray(int64/uint64) or integer) – Value(s) to rotate left.
rot (pdarray(int64/uint64) or integer) – Amount(s) to rotate by.

Returns:

rotated – The rotated elements of x.

Return type:

pdarray(int64/uint64)

Raises:

TypeError – If input array is not int64 or uint64

Examples

>>> A = ak.arange(10)
>>> ak.rotl(A, A)
array([0, 2, 8, 24, 64, 160, 384, 896, 2048, 4608])

arkouda.pdarrayclass.rotr(x, rot) → pdarray[source]¶

Rotate bits of <x> to the left by <rot>.

Parameters:

x (pdarray(int64/uint64) or integer) – Value(s) to rotate left.
rot (pdarray(int64/uint64) or integer) – Amount(s) to rotate by.

Returns:

rotated – The rotated elements of x.

Return type:

pdarray(int64/uint64)

Raises:

TypeError – If input array is not int64 or uint64

Examples

>>> A = ak.arange(10)
>>> ak.rotr(1024 * A, A)
array([0, 512, 512, 384, 256, 160, 96, 56, 32, 18])

arkouda.pdarrayclass.sqrt(pda: pdarray, where: arkouda.numpy.dtypes.bool_scalars | pdarray = True) → pdarray[source]¶

Takes the square root of array. If where is given, the operation will only take place in the positions where the where condition is True.

Parameters:

pda (pdarray) – A pdarray of values that will be square rooted
where (Boolean or pdarray) – This condition is broadcast over the input. At locations where the condition is True, the corresponding value will be square rooted. Elsewhere, it will retain its original value. Default set to True.

Returns:

pdarray Returns a pdarray of square rooted values, under the boolean where condition.
Examples
>>> a = ak.arange(5)
>>> ak.sqrt(a)
array([0 1 1.4142135623730951 1.7320508075688772 2])
>>> ak.sqrt(a, ak.sqrt([True, True, False, False, True]))
array([0, 1, 2, 3, 2])

arkouda.pdarrayclass.std(pda: pdarray, ddof: arkouda.numpy.dtypes.int_scalars = 0) → numpy.float64[source]¶

Return the standard deviation of values in the array. The standard deviation is implemented as the square root of the variance.

Parameters:

pda (pdarray) – values for which to calculate the standard deviation
ddof (int_scalars) – “Delta Degrees of Freedom” used in calculating std

Returns:

The scalar standard deviation of the array

Return type:

np.float64

Raises:

TypeError – Raised if pda is not a pdarray instance or ddof is not an integer
ValueError – Raised if ddof is an integer < 0
RuntimeError – Raised if there’s a server-side error thrown

See also

mean, var

Notes

The standard deviation is the square root of the average of the squared deviations from the mean, i.e., std = sqrt(mean((x - x.mean())**2)).

The average squared deviation is normally calculated as x.sum() / N, where N = len(x). If, however, ddof is specified, the divisor N - ddof is used instead. In standard statistical practice, ddof=1 provides an unbiased estimator of the variance of the infinite population. ddof=0 provides a maximum likelihood estimate of the variance for normally distributed variables. The standard deviation computed in this function is the square root of the estimated variance, so even with ddof=1, it will not be an unbiased estimate of the standard deviation per se.

arkouda.pdarrayclass.unregister_pdarray_by_name(user_defined_name: str) → None[source]¶

Unregister a named pdarray in the arkouda server which was previously registered using register() and/or attahced to using attach_pdarray()

Parameters:: user_defined_name (str) – user defined name which array was registered under
Return type:: None
Raises:: RuntimeError – Raised if the server could not find the internal name/symbol to remove

See also

register, unregister, is_registered, list_registry, attach

Notes

Registered names/pdarrays in the server are immune to deletion until they are unregistered.

Examples

>>> a = zeros(100)
>>> a.register("my_zeros")
>>> # potentially disconnect from server and reconnect to server
>>> b = ak.attach_pdarray("my_zeros")
>>> # ...other work...
>>> ak.unregister_pdarray_by_name(b)

arkouda.pdarrayclass.var(pda: pdarray, ddof: arkouda.numpy.dtypes.int_scalars = 0) → numpy.float64[source]¶

Return the variance of values in the array.

Parameters:

pda (pdarray) – Values for which to calculate the variance
ddof (int_scalars) – “Delta Degrees of Freedom” used in calculating var

Returns:

The scalar variance of the array

Return type:

np.float64

Raises:

TypeError – Raised if pda is not a pdarray instance
ValueError – Raised if the ddof >= pdarray size
RuntimeError – Raised if there’s a server-side error thrown

See also

mean, std

Notes

The variance is the average of the squared deviations from the mean, i.e., var = mean((x - x.mean())**2).

The mean is normally calculated as x.sum() / N, where N = len(x). If, however, ddof is specified, the divisor N - ddof is used instead. In standard statistical practice, ddof=1 provides an unbiased estimator of the variance of a hypothetical infinite population. ddof=0 provides a maximum likelihood estimate of the variance for normally distributed variables.