Skip to content

[Bug]: NWB Zarr to HDMF export fails #211

@rcpeene

Description

@rcpeene

What happened?

Trying to export a zarr nwb as hdmf, but it yields an error

Steps to Reproduce

Running the following snippet on an nwb file

    with NWBZarrIO(str(zarr_filename), mode='r') as read_io:  # Create Zarr IO object for read
        with NWBHDF5IO(hdmf_filename, 'w') as export_io:  # Create HDF5 IO object for write
            export_io.export(src_io=read_io, write_args=dict(link_data=False))  # Export from Zarr to HDF5

I can't share the nwb file for licensing reasons



### Traceback

```shell
/opt/conda/lib/python3.9/site-packages/hdmf/common/table.py:489: UserWarning: An attribute 'name' already exists on DynamicTable 'eye_tracking' so this column cannot be accessed as an attribute, e.g., table.name; it can only be accessed using other methods, e.g., table['name'].
  self.__set_table_attr(col)
Traceback (most recent call last):
  File "/root/capsule/./code/run_capsule.py", line 57, in <module>
    if __name__ == "__main__": run()
  File "/root/capsule/./code/run_capsule.py", line 47, in run
    export_io.export(src_io=read_io, write_args=dict(link_data=False))  # Export from Zarr to HDF5
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/pynwb/__init__.py", line 399, in export
    super().export(**kwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 458, in export
    super().export(**ckwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/io.py", line 166, in export
    self.write_builder(builder=bldr, **write_args)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 836, in write_builder
    self.write_group(self.__file, gbldr, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 1018, in write_group
    self.write_group(group, sub_builder, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 1023, in write_group
    self.write_dataset(group, sub_builder, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 1326, in write_dataset
    dset = self.__list_fill__(parent, name, data, options)
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 1492, in __list_fill__
    raise e
  File "/opt/conda/lib/python3.9/site-packages/hdmf/backends/hdf5/h5tools.py", line 1490, in __list_fill__
    dset[:] = data
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/opt/conda/lib/python3.9/site-packages/h5py/_hl/dataset.py", line 898, in __setitem__
    val = numpy.asarray(val, dtype=dtype.base, order='C')
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 579, in __array__
    a = self[...]
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 800, in __getitem__
    result = self.get_basic_selection(pure_selection, fields=fields)
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 926, in get_basic_selection
    return self._get_basic_selection_nd(selection=selection, out=out, fields=fields)
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 968, in _get_basic_selection_nd
    return self._get_selection(indexer=indexer, out=out, fields=fields)
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 1343, in _get_selection
    self._chunk_getitems(
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 2183, in _chunk_getitems
    self._process_chunk(
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 2096, in _process_chunk
    chunk = self._decode_chunk(cdata)
  File "/opt/conda/lib/python3.9/site-packages/zarr/core.py", line 2366, in _decode_chunk
    chunk = chunk.view(self._dtype)
ValueError: When changing to a smaller dtype, its size must be a divisor of the size of original dtype

Operating System

Linux

Python Executable

Python

Python Version

3.9

Package Versions

No response

Code of Conduct

Metadata

Metadata

Assignees

Labels

category: bugerrors in the code or code behaviorpriority: mediumnon-critical problem and/or affecting only a small set of users

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions