Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create base data layer #1592

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

feat: create base data layer #1592

wants to merge 17 commits into from

Conversation

desaxce
Copy link
Collaborator

@desaxce desaxce commented Dec 18, 2024

Changes:

  • add ChainlitDataLayer which logs automatically thread/steps/user/feedback to DB with schema aligning with https://github.com/Chainlit/chainlit-datalayer
  • adjust S3 / Azure / GCS storage clients to support element (attachment) uploads

To test:

I didn't test with Azure.

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. backend Pertains to the Python backend. data layer Pertains to data layers. enhancement New feature or request labels Dec 18, 2024
:type, :start_time, :end_time, :variables, :settings, :tools,
:root_run_id
)
ON CONFLICT (id) DO UPDATE SET
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to go with on conflict because a step might already exist in DB, just not yet populated.

@@ -79,3 +80,50 @@ async def upload_file(
except Exception as e:
logger.warn(f"AzureStorageClient, upload_file error: {e}")
return {}


class AzureBlobStorageClient(BaseStorageClient):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating a separate Azure client for blobs as it seems more appropriate to cloud storage for attachments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Pertains to the Python backend. data layer Pertains to data layers. enhancement New feature or request size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants