awkward.fromiter is the one function that was allowed to be written in Python for loops, and hence it is designated to be slow. The function you want for turning regular NumPy arrays into JaggedArrays that happen to have uniform counts is JaggedArray.fromregular. That ought to be considerably faster.

Meanwhile, your original issue is an example of an inconsistency in Awkward 0.x. In Awkward 1.x, the behavior of Awkward Arrays that happen to be regular and NumPy arrays with the same logical meaning are identical. You might want to consider awkward1.from_awkward0 in the awkward1 library to try it out. (It's a separate library because the interface is a little different and I don't want to break anyone's analysis!)

Answer from Jim Pivarski on Stack Overflow
🌐
Awkward-array
awkward-array.org › doc › main › reference › generated › ak.mask.html
ak.mask — Awkward Array 2.13.0 documentation
>>> ak.mask(array, good) + array <Array [None, 2, None, 6, None, 10, None, 14, None, 18] type='10 * ?int64'>
🌐
Awkward-array
awkward-array.org › doc › main › user-guide › how-to-filter-cut-mask.html
How to filter arrays: cutting vs. masking — Awkward Array 2.9.0 documentation
83 if self.primary() is self: File ~/micromamba/envs/awkward-docs/lib/python3.11/site-packages/awkward/highlevel.py:1633, in Array.__array_ufunc__(self, ufunc, method, *inputs, **kwargs) 1631 name = f"{type(ufunc).__module__}.{ufunc.__name__}.{method!s}" 1632 with ak._errors.OperationErrorContext(name, inputs, kwargs): -> 1633 return ak._connect.numpy.array_ufunc(ufunc, method, inputs, kwargs) File ~/micromamba/envs/awkward-docs/lib/python3.11/site-packages/awkward/_connect/numpy.py:485, in array_ufunc(ufunc, method, inputs, kwargs) 477 raise TypeError( 478 "no {}.{} overloads for custom types
🌐
GitHub
github.com › scikit-hep › awkward-0.x
GitHub - scikit-hep/awkward-0.x: Manipulate arrays of complex data structures as easily as Numpy. · GitHub
Naturally, the only kinds of arrays Numpy can mask are subclasses of its own ndarray, and we need to be able to mask any Awkward Array, so the Awkward library defines its own MaskedArray.
Starred by 214 users
Forked by 38 users
Languages: Python 63.7% | Jupyter Notebook 36.3%
🌐
Cms-opendata-workshop
cms-opendata-workshop.github.io › workshop2022-lesson-cpp-root-python › 08-awkward
Using awkward arrays to analyze HEP data – ROOT with C++ and python
August 2, 2022 - A closer looks shows us that while pt is a numpy array, the elements are not arrays but python list objects, which behave differently. The reason this happened is that numpy arrays can’t deal with this type of ``jagged” behavior where the first row of your data might have 2 elements and the second row might have 3 elements and the third row might have 0 elements and so on. For that, we need awkward-array.
🌐
Awkward-array
awkward-array.org › how-to-create-missing.html
How to create arrays of missing data
Data at any level of an Awkward Array can be “missing,” represented by None in Python. This functionality is somewhat like NumPy’s masked arrays, but masked arrays can only declare numerical values to be missing (not, for instance, a row of a 2-dimensional array) and they represent missing ...
Find elsewhere
🌐
Awkward-array
awkward-array.org › doc › main › _sources › user-guide › how-to-convert-numpy.md.txt
--- jupytext: text_representation: extension: .md format_name: myst
NumPy's masked arrays correspond to Awkward's "option type." {code-cell} ipython3 np_array = np.ma.MaskedArray( [[1, 2, 3], [4, 5, 6]], mask=[[False, True, False], [True, True, False]] ) np_array · {code-cell} ipython3 np_array.tolist() {code-cell} ipython3 ak_array = ak.from_numpy(np_array) ak_array · The ? before int64 (expands to option[...] for more complex contents) refers to "option type," meaning that the values can be missing ("None" in Python).
🌐
Readthedocs
awkward-array.readthedocs.io › en › latest › _auto › ak.Array.html
ak.Array — Awkward Array documentation
See filtering and ak.mask. ... Converts this Array into Python objects; same as ak.to_list (but without the underscore, like NumPy’s tolist).
🌐
GitHub
github.com › scikit-hep › awkward › issues › 1459
Masking and binning awkward.highlevel.Array leads to wrong results · Issue #1459 · scikit-hep/awkward
May 5, 2022 - In addition when making the histogram of the masked array it looks like that the masked values are not removed before making the histogram (while they are removed when using other numpy functions like np.sum). I made a short video to illustrate the issue. ... import numpy as np import awkward as ak import pickle import sys print(sys.version, ak.__version__, np.__version__) # Load and prepare data: with open("test_data_awkward_numpy_bug", "rb") as fp: # Unpickling list_root = pickle.load(fp) test_data = ak.Array(list_root) first_photon = ak.min(test_data['pmthitTime'], axis=1) first_photon = first_photon * 10**9 test_mask = (test_data['nNVpmthits'] >= 300 ) & (test_data['nNVpmthits'] < 700) test_mask = test_mask & (first_photon < 20) # Bug observed in awkward arrays using numpy.ma.core.MaskedArray # No sure if the bug is in awkward or numpy.
Author: scikit-hep
🌐
Awkward-array
awkward-array.org › doc › main › reference › ak.behavior.html
ak.behavior — Awkward Array 2.14.0 documentation
Custom reducers are invoked with ... an option type (for reductions along empty sublists). If the reducer does not introduce an option type, and mask=True, Awkward will mask the result at the appropriate positions....
🌐
Awkward-array
awkward-array.org › doc › main › reference › generated › ak.Array.html
ak.Array — Awkward Array 2.13.0 documentation
See filtering and ak.mask. ... Converts this Array into Python objects; same as ak.to_list (but without the underscore, like NumPy’s tolist).
🌐
Stack Overflow
stackoverflow.com › questions › tagged › awkward-array
Newest 'awkward-array' Questions - Stack Overflow
I am using awkward/2.6.7, awkward_cpp/37, jupyterlab==3.0.... ... I have a numpy array I converted from awkward array by to_numpy() function, and the resulting array has the datatype: dtype=[('phi', '<f8'), ('eta', '<f8')]).
🌐
GitHub
github.com › scikit-hep › awkward › discussions › 633
Merging arrays based on common fields (i.e. "joining" tables) · scikit-hep/awkward · Discussion #633
January 4, 2021 - Since Awkward Array has missing values (more generally than NumPy's masked arrays), we've introduced another kind of filtering operation, array.mask[filter], that does not change the length of the array; it masks out values by replacing them with a placeholder None.
Author: scikit-hep
🌐
Awkward-array
awkward-array.org › doc › main › user-guide › how-to-filter-ragged.html
How to filter with ragged arrays — Awkward Array 2.13.0 documentation
Ufuncs, such as np.mod(), are powerful tools for generating boolean masks, as they directly preserve the exact structure of the original array:
🌐
GitHub
github.com › pfackeldey › awkward-array
GitHub - pfackeldey/awkward-array: Manipulate arrays of complex data structures as easily as Numpy.
Internally, the data has been rearranged into a columnar form, with all values at a given level of hierarchy in the same array. Numpy-like slicing, masking, and fancy indexing are translated into Numpy operations on these internal arrays: they are not implemented with Python for loops!
Author: pfackeldey
🌐
Snyk
snyk.io › advisor › awkward › functions › awkward.array.table.table
How to use the awkward.array.table.Table function in awkward | Snyk
scikit-hep / awkward-array / awkward / arrow.py View on Github · return recurse(obj.array, mask) else: raise TypeError("cannot convert type {0} to Arrow".format(type(obj))) if isinstance(obj, awkward.array.chunked.ChunkedArray): # includes AppendableArray batches = [] for chunk in obj.chunks: arr = toarrow(chunk) if isinstance(arr, pyarrow.Table): batches.extend(arr.to_batches()) else: batches.append(pyarrow.RecordBatch.from_arrays([arr], [""])) return pyarrow.Table.from_batches(batches) elif isinstance(obj, awkward.array.masked.IndexedMaskedArray) and isinstance(obj.content, awkward.array.ta