You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
'DataFrame' object has no attribute 'to_msgpack'
Traceback (most recent call last):
File "<...>/timeflux/core/worker.py", line 57, in _run
scheduler.run()
File "<...>//timeflux/core/scheduler.py", line 22, in run
self.next()
File "<...>//timeflux/core/scheduler.py", line 59, in next
self._nodes[step["node"]].update()
File "<...>//timeflux/nodes/zmq.py", line 177, in update
self._socket.send_serialized(
File "<...>//site-packages/zmq/sugar/socket.py", line 854, in send_serialized
frames = serialize(msg)
File "<...>//timeflux/core/message.py", line 31, in msgpack_serialize
return [topic, data.to_msgpack()]
File "/home/arne/.virtualenvs/timeflux-workshop/lib64/python3.9/site-packages/pandas/core/generic.py", line 5989, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'to_msgpack'
Outputs from Timeflux nodes are objects consisting of two components:
A DataFrame structure with a Pandas-compatible API ;
An arbitrary Python dictionary containing metadata such as sample rate, context, etc.
So we still need a way to serialize the metadata. We could use Pickle, but that would limit interoperability to Python applications. Metadata can contain complex datatypes that could be more efficiently handled by libraries such as orjson, but we would introduce another dependency. Would a simple JSON serialization using the standard library work for you? If so, we could just log a warning if metadata (de)serialization fails.
If that sounds like an acceptable solution for you, feel free to open a PR!
Steps to reproduce
When creating a
timeflux.nodes.zmq.Pub
node with themsgpack
serializer liketimeflux errors with
Issue
When initialized using
serialize='msgpack
ordeserialize='msgpack'
,timeflux.nodes.zmq.Pub
andtimeflux.nodes.zmq.Sub
respectively retrievetimeflux.core.message.msgpack_serialize
andtimeflux.core.message.msgpack_deserialize
methods, which in turn callpandas.DataFrame.to_msgpack
andpandas.read_msgpack
. These methods have been deprecated since Pandas 0.25 and were removed in newer versions:https://pandas.pydata.org/pandas-docs/version/0.25.0/reference/api/pandas.DataFrame.to_msgpack.html
https://pandas.pydata.org/pandas-docs/version/0.25.3/reference/api/pandas.read_msgpack.html
Proposed fixes
timeflux.core.message
to use the equivalent methods from pandas-msgpack (also no longer maintained)The text was updated successfully, but these errors were encountered: