You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description="A JSON-formatted string representing the value to set for the XCom.",
openapi_examples={
"simple_value": {
"summary": "Simple value",
"value": '"value1"',
},
"dict_value": {
"summary": "Dictionary value",
"value": '{"key2": "value2"}',
},
"list_value": {
"summary": "List value",
"value": '["value1"]',
},
},
),
],
This was done so that we can have the same interface for multi-language support (Python, Go, Java etc) for Task SDK. The contract is simple: API Server always deals with JSON-formatted string --- and the responsibility of serialization and de-serialization of Native objects to string and back lies to the language-specific clients.
In order to do that we should move the current Serialization and de-serialization logic to clients. This will also allow storing and using XCom backends correctly, which can also handle different languages.
As part of this task, we should also figure out how we can avoid serialization and deserialization from SQLAlchemy as we use JSON type for value column --- which leads to double / triple serialization
Serialization from Task SDK and sent to API Server
API Server calls XCom.serialize
SQLAlchemy JSON type serializes it further
Result of Triple serialization:
The text was updated successfully, but these errors were encountered:
Currently, as part of AIP-72, we implemented XCom API endpoints in the Task Execution API Server that expect a JSON-serialized string
airflow/airflow/api_fastapi/execution_api/routes/xcoms.py
Lines 114 to 138 in 9be5971
This was done so that we can have the same interface for multi-language support (Python, Go, Java etc) for Task SDK. The contract is simple: API Server always deals with JSON-formatted string --- and the responsibility of serialization and de-serialization of Native objects to string and back lies to the language-specific clients.
In order to do that we should move the current Serialization and de-serialization logic to clients. This will also allow storing and using XCom backends correctly, which can also handle different languages.
As part of this task, we should also figure out how we can avoid serialization and deserialization from SQLAlchemy as we use JSON type for
value
column --- which leads to double / triple serializationJSON
type serializes it furtherResult of Triple serialization:
The text was updated successfully, but these errors were encountered: