nested pydantic cloudevent serialization doesn't work #198

Activity

1. xSAVIKx commented on Nov 9, 2022

   

   xSAVIKx

   on Nov 9, 2022

   Member

   More actions

   that's kinda the "default" behavior, but sure, let's wait for a PR 👍
2. hozn commented on Jan 5, 2024

   

   hozn

   on Jan 5, 2024

   More actions

   I'm having this issue as well, but in pydantic v2. It works fine in vanilla pydantic, but does not work with the CloudEvent -- or subclasses, as I am using.

   In my specific case, I have a Generic subclass for our models.

   import uuid
   from typing import Generic, TypeVar
   
   from cloudevents.pydantic.v2 import CloudEvent
   from pydantic import BaseModel
   
   T = TypeVar("T", bound=BaseModel)
   
   
   class MyEntity(BaseModel):
       id: uuid.UUID
   
   
   class MyCloudEvent(CloudEvent, Generic[T]):
       data: T
   
   
   if __name__ == "__main__":
       obj = MyEntity(id=uuid.uuid4())
   
       evt = MyCloudEvent(
           id=str(uuid.uuid4()),
           source="urn:source",
           subject="subject",
           data=obj,
           event_type="MyEntity",
           type="xyz",
       )
       # This works:
       # print(evt.model_dump(mode="python"))
   
       # But this does not:
       print(evt.model_dump(mode="json"))

   Stack trace snippet:

   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/hans/Library/Application Support/JetBrains/PyCharm2023.2/scratches/scratch_42.py", line 30, in <module>
       print(evt.model_dump(mode="json"))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File "/Users/hans/Library/Caches/pypoetry/virtualenvs/cvPU66vz-py3.11/lib/python3.11/site-packages/pydantic/main.py", line 308, in model_dump
       return self.__pydantic_serializer__.to_python(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   pydantic_core._pydantic_core.PydanticSerializationError: Error calling function `_ce_json_dumps`: TypeError: Object of type MyEntity is not JSON serializable
   
3. hozn commented on Jan 5, 2024

   

   hozn

   on Jan 5, 2024

   More actions

   We're using fastapi, so the workaround I'm using for the above nested-object serialization issue is:

   from fastapi.encoders import jsonable_encoder
   
   jsonable_encoder(event.model_dump(mode="python"))

   Which works ... but this used to work without these extra hoops in pydantic v1 (and older cloudevents lib).
4. alex-schaaf commented on Nov 29, 2024

   

   alex-schaaf

   on Nov 29, 2024

   More actions

   We also just ran into the same problem with pydantic v2 and cloudevents 1.11.0. We want to serialize the CloudEvent pydantic model (with a pydantic model as the data) to json before sending it out, so that we ensure that StrEnum values are properly converted to their values, but it crashes with the same error message:

   event.model_dump_json()
              ^^^^^^^^^^^^^^^^^^^^^^^
     File "/Users/*/Library/Caches/pypoetry/virtualenvs/*/lib/python3.12/site-packages/pydantic/main.py", line 441, in model_dump_json
       return self.__pydantic_serializer__.to_json(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Error calling function `_ce_json_dumps`: TypeError: Object of type Payload is not JSON serializable
   

   We now did a dirty workaround of first serializing the event data to json, then loading it back into a Python dictionary and the processing it further, before we emit our event.

   👍React with 👍2Reacted by Ronald and Thomas Hausberger
5. black-snow commented on Nov 29, 2024

   

   black-snow

   on Nov 29, 2024

   More actions

   We now did a dirty workaround of first serializing the event data to json, then loading it back into a Python dictionary and the processing it further, before we emit our event.

   Which is kinda silly because _ce_json_dumps basically does the same thing. So in the end the object gets json encoded and decoded several times, which is quite wasteful (and potentially prone to the bad kind of bugs).

   👍React with 👍1Reacted by Alexander Schaaf