You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently if you want to use pyroscope the only way to do that is to start a pyroscope server (acts as a db and a UI), and then separately start an agent process that would collect profiling data from the target app and send it to the server.
This works great for continuous profiling in production use case. However, some people have been asking us for an adhoc mode in which you would be able to connect to a running process, profile it and then immediately view the results.
Example
If we have a script, lets call it script.py. Right now, in order to profile this script we would have to do the following:
Start the server: pyroscope server
Start the agent (and profile the script) pyroscope exec script.py
For a server this flow makes sense. However for a script, we wouldn't need the Pyroscope server to continue running and it should really be one quick command: pyroscope adhoc script.py
The frontend of these two methodologies would be indistinguishable from each other, but leave room for future improvements (see section below).
Scope
Step 1
For now we want to start with the most basic implementation of such feature. We could combine the pyroscope exec subcommand with pyroscope server subcommand in one process. We could call it pyroscope adhoc. This command would start a server, start the agent and then connect the two together. Note: this may be slightly different for Go than other languages, lets start with Ruby and Python first
pyroscope adhoc test.py
pyroscope adhoc test.rb
Step 2 (likely separate PR)
After the server and agent are started together using pyroscope adhoc there are two potential ways to proceed:
Export the data into an .html similar to rbspy, pyspy, pprof etc.
Continue to keep the server running and visualize the data in our normal UI
For 1 above, a potential route would be reusing our compiled webpack bundle (or something along these lines).
For 2 above, the idea would be to possibly add a new "tab" that is able to read previous saved htmls to show multiple different "runs" of pyroscope adhoc on scripts
We can talk about this step once Step 1 is completed but just added this for context. Also see "next steps" below for more context about future potential.
Technical Details
Server requires a path to a directory where the data would be stored. I'm pretty sure there's an in-memory mode for badger (the db we use), so we could just use that potentially.
Currently the agent code has a Session. This session collects profiles and sends them to an Upstream. There's two implementations for this upstream interface — Remote and Direct. By default agent uses remote interface (remote means it sends data over http). If we switched to Direct implementation it would allow us to write data directly to badger within the same process.
Potential Future Improvements
There's a lot of cool things we could build on top of this, e.g:
store all profiles locally at something like ~/.pyroscope
add a concept of a "run" (similar to a trace I guess)
have ability to export runs to a standalone svg / html file that people could share later.
have annotations on the timeline so that people can search for these runs effectively
@Rperry2174 has more ideas on what we could do with this too.
The text was updated successfully, but these errors were encountered:
Adding this as a mockup for what a potential end UI would look like.
For continuous profiling, the input is the "start time" and "end time" which comes from the timeline. This makes sense for continuous profiling because part of the appeal here is being able to drill down into particular time ranges.
With adhoc profiling the idea is to be able to take a file as input and then output a flamegraph. Then over time, you can revisit previous ad-hoc profiles that you have created via a file-selector-type-system. This would allow us to recreate all three of our main views:
single view
comparison view
diff view
By simply replacing the timeline input component, with a file-selector input component.
We likely would output the files to a specific folder so this mockup is slightly inaccurate in that sense, but the general idea is there.
Summary
Currently if you want to use pyroscope the only way to do that is to start a pyroscope server (acts as a db and a UI), and then separately start an agent process that would collect profiling data from the target app and send it to the server.
This works great for continuous profiling in production use case. However, some people have been asking us for an adhoc mode in which you would be able to connect to a running process, profile it and then immediately view the results.
Example
If we have a script, lets call it
script.py
. Right now, in order to profile this script we would have to do the following:pyroscope server
pyroscope exec script.py
For a server this flow makes sense. However for a script, we wouldn't need the Pyroscope server to continue running and it should really be one quick command:
pyroscope adhoc script.py
The frontend of these two methodologies would be indistinguishable from each other, but leave room for future improvements (see section below).
Scope
Step 1
For now we want to start with the most basic implementation of such feature. We could combine the
pyroscope exec
subcommand withpyroscope server
subcommand in one process. We could call itpyroscope adhoc
. This command would start a server, start the agent and then connect the two together.Note: this may be slightly different for Go than other languages, lets start with Ruby and Python first
pyroscope adhoc test.py
pyroscope adhoc test.rb
Step 2 (likely separate PR)
After the server and agent are started together using
pyroscope adhoc
there are two potential ways to proceed:.html
similar to rbspy, pyspy, pprof etc.For 1 above, a potential route would be reusing our compiled webpack bundle (or something along these lines).
For 2 above, the idea would be to possibly add a new "tab" that is able to read previous saved htmls to show multiple different "runs" of
pyroscope adhoc
on scriptsWe can talk about this step once Step 1 is completed but just added this for context. Also see "next steps" below for more context about future potential.
Technical Details
Server requires a path to a directory where the data would be stored. I'm pretty sure there's an in-memory mode for badger (the db we use), so we could just use that potentially.
Currently the agent code has a
Session
. This session collects profiles and sends them to anUpstream
. There's two implementations for this upstream interface —Remote
andDirect
. By default agent uses remote interface (remote means it sends data over http). If we switched toDirect
implementation it would allow us to write data directly to badger within the same process.Potential Future Improvements
There's a lot of cool things we could build on top of this, e.g:
~/.pyroscope
@Rperry2174 has more ideas on what we could do with this too.
The text was updated successfully, but these errors were encountered: