add numpy array to data container (ict.data.add) - copy vs. view
I supply data as
ict.data.add(arr)
# with
type(arr)
Out[20]: numpy.ndarray
arr.shape
Out[21]: (86677, 3)
arr.dtype
Out[22]: dtype('float64')
But it fails as soon as certain variables are called by key, e.g. if I call ict.write
, which then tries to convert np.nan to vmiss:
Traceback (most recent call last):
File "C:\Users\va6504\AppData\Local\Temp\ipykernel_16448\2416392629.py", line 1, in <cell line: 1>
ict.write()
File "D:\PROGRAMMING\Python\icartt_dev\src\icartt\dataset.py", line 876, in write
self.writeData(f=f, fmt=fmt, delimiter=delimiter)
File "D:\PROGRAMMING\Python\icartt_dev\src\icartt\dataset.py", line 865, in writeData
self.data.write(f=f, fmt=fmt, delimiter=delimiter)
File "D:\PROGRAMMING\Python\icartt_dev\src\icartt\dataset.py", line 104, in write
d = self.denanify(self.data)
File "D:\PROGRAMMING\Python\icartt_dev\src\icartt\dataset.py", line 97, in denanify
dd[k][np.isnan(dd[k])] = miss
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
After further inspection, I found that the problem was that I transposed the array before; np.transpose returns a view, not a copy. If I feed ict.data.add(arr.copy())
, everything works fine.
So I was thinking if we might add a check if the user is passing a view or a copy. Reading How can I tell if NumPy creates a view or a copy? and this linked question, only .flags['OWNDATA']
seems like a viable option to me. But not sure how reliable that is for given question. In any case, it must be clear from the docs that ict.data.add
expects a copy, not a view.
Edited by Florian Obersteiner