Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Centralized Request Processing middleware #3847

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

dave-gray101
Copy link
Collaborator

@dave-gray101 dave-gray101 commented Oct 16, 2024

This PR reworks how our endpoints parse and process user input, in order to centralize and commonize that code as much as possible. This will drastically cut down the endpoint-specific code to a minimum.

Additionally, the VAD subsystem was untested - making it hard to validate if this changeset was safe or not. Therefore, this PR also adds both manual testing tools (via bruno) and automated testing in the AIO image.


Note to @mudler - this PR looks worse than it is - here's some reviewing notes to help

  • core/http/middleware/request.go should be where you start - its the heart of the change and replaces core/http/ctx/fiber.go
  • configuration of defaults is now a responsibility of the route function - seems a bit weird at first, but allows endpoints to be easily reused with different defaults on different paths.
  • while all the endpoint files were updated, the changes all follow the same pattern:
    • A new standard header to the function was added which pulls the relevant input and config from the earlier middleware stage
    • A bunch of code is deleted as its now performed earlier by those middlewares
    • The functional code specific to the endpoint is unchanged except for variable references
  • Specific to the AIO CI test: I was seeing very unpredictable behavior on CI versus locally reproducing this test. It turns out that setting LOCALAI_SINGLE_ACTIVE_BACKEND=true helps with a memory usage issue!

Copy link

netlify bot commented Oct 16, 2024

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 2623eaa
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/676f3acbb064e50008d55fe9
😎 Deploy Preview https://deploy-preview-3847--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@dave-gray101
Copy link
Collaborator Author

dave-gray101 commented Nov 21, 2024

@mudler merged in VAD, but I don't know how to test it manually yet since it's new. Please double check that :)

@dave-gray101 dave-gray101 force-pushed the feat-request-middleware branch from df37c12 to d08311f Compare December 17, 2024 20:06
@dave-gray101
Copy link
Collaborator Author

@mudler Ready for review, now with working VAD test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant