Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bin: config: Provide windows.maxstdio option for increasing I/O limit on Windows #9707

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Dec 11, 2024

With default limit of I/O on Windows is somewhat small for large amount of log volume environment.
I provide an option to set up this limit up to 2048.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

fluent/fluent-bit-docs#1528

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@cosmo0920 cosmo0920 force-pushed the cosmo0920-provide-maxstdio-option-on-windows branch from ccd1d52 to c5a552f Compare December 12, 2024 06:51
@cosmo0920 cosmo0920 force-pushed the cosmo0920-provide-maxstdio-option-on-windows branch from c5a552f to cc0a9e3 Compare December 12, 2024 07:57
@cosmo0920 cosmo0920 marked this pull request as ready for review December 12, 2024 08:09
@cosmo0920 cosmo0920 added this to the Fluent Bit v3.2.3 milestone Dec 12, 2024
@edsiper
Copy link
Member

edsiper commented Dec 19, 2024

@cosmo0920 we don't do ulimit stuff in Fluent Bit, why should we do it for Windows ? (maybe we just need to update the docs for windows users ?)

@nuclearpidgeon
Copy link
Contributor

nuclearpidgeon commented Dec 20, 2024

Is this limit something that can actually be changed at a system configuration level on Windows?

This is the MS doco about the setter function: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmaxstdio?view=msvc-170 . It seems to be specifically something that's part of the Microsoft C runtime library (CRT).

There's some info here saying that the limit can be further stretched if you use specific Win32 calls directly rather than this C runtime library: https://stackoverflow.com/questions/870173/is-there-a-limit-on-number-of-open-files-in-windows/4276338#4276338

@cosmo0920
Copy link
Contributor Author

@cosmo0920 we don't do ulimit stuff in Fluent Bit, why should we do it for Windows ? (maybe we just need to update the docs for windows users ?)

In Windows, we need to set up the ulimit like limit via _setmaxstdio. There is no API/mechanism without this API from C runtime.

@edsiper
Copy link
Member

edsiper commented Dec 29, 2024

thanks folks for the context. @cosmo0920 I suggest to rename this option to windows.maxstdio

@cosmo0920
Copy link
Contributor Author

Sure, I'll rename it from the previous one.

@cosmo0920 cosmo0920 force-pushed the cosmo0920-provide-maxstdio-option-on-windows branch from cc0a9e3 to 7c0b614 Compare December 30, 2024 14:36
@cosmo0920 cosmo0920 changed the title bin: config: Provide maxstdio option for increasing I/O limit on Windows bin: config: Provide windows.maxstdio option for increasing I/O limit on Windows Dec 30, 2024
@cosmo0920
Copy link
Contributor Author

I rebased off the current master and added windows. prefix on the newly introduced parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants