Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for ad-hoc profiling. #228

Closed
2 tasks
petethepig opened this issue Jun 1, 2021 · 2 comments
Closed
2 tasks

Add support for ad-hoc profiling. #228

petethepig opened this issue Jun 1, 2021 · 2 comments
Assignees
Labels
backend Mostly go code

Comments

@petethepig
Copy link
Member

petethepig commented Jun 1, 2021

Summary

Currently if you want to use pyroscope the only way to do that is to start a pyroscope server (acts as a db and a UI), and then separately start an agent process that would collect profiling data from the target app and send it to the server.

This works great for continuous profiling in production use case. However, some people have been asking us for an adhoc mode in which you would be able to connect to a running process, profile it and then immediately view the results.

Example

If we have a script, lets call it script.py. Right now, in order to profile this script we would have to do the following:

  1. Start the server: pyroscope server
  2. Start the agent (and profile the script) pyroscope exec script.py

For a server this flow makes sense. However for a script, we wouldn't need the Pyroscope server to continue running and it should really be one quick command:
pyroscope adhoc script.py
The frontend of these two methodologies would be indistinguishable from each other, but leave room for future improvements (see section below).

Scope

Step 1

For now we want to start with the most basic implementation of such feature. We could combine the pyroscope exec subcommand with pyroscope server subcommand in one process. We could call it pyroscope adhoc. This command would start a server, start the agent and then connect the two together.
Note: this may be slightly different for Go than other languages, lets start with Ruby and Python first

  • pyroscope adhoc test.py
  • pyroscope adhoc test.rb

Step 2 (likely separate PR)

After the server and agent are started together using pyroscope adhoc there are two potential ways to proceed:

  1. Export the data into an .html similar to rbspy, pyspy, pprof etc.
  2. Continue to keep the server running and visualize the data in our normal UI

For 1 above, a potential route would be reusing our compiled webpack bundle (or something along these lines).
For 2 above, the idea would be to possibly add a new "tab" that is able to read previous saved htmls to show multiple different "runs" of pyroscope adhoc on scripts

We can talk about this step once Step 1 is completed but just added this for context. Also see "next steps" below for more context about future potential.

Technical Details

Server requires a path to a directory where the data would be stored. I'm pretty sure there's an in-memory mode for badger (the db we use), so we could just use that potentially.

Currently the agent code has a Session . This session collects profiles and sends them to an Upstream. There's two implementations for this upstream interface — Remote and Direct. By default agent uses remote interface (remote means it sends data over http). If we switched to Direct implementation it would allow us to write data directly to badger within the same process.

Potential Future Improvements

There's a lot of cool things we could build on top of this, e.g:

  • store all profiles locally at something like ~/.pyroscope
  • add a concept of a "run" (similar to a trace I guess)
  • have ability to export runs to a standalone svg / html file that people could share later.
  • have annotations on the timeline so that people can search for these runs effectively

@Rperry2174 has more ideas on what we could do with this too.

@eh-am
Copy link
Collaborator

eh-am commented Oct 14, 2021

Here's an example of workflow that the adhoc functionality should provide (ignoring that's that's a nodejs repo)

mapbox/flamebearer#7 (comment)

@Rperry2174
Copy link
Contributor

Adding this as a mockup for what a potential end UI would look like.
adhoc_flow

For continuous profiling, the input is the "start time" and "end time" which comes from the timeline. This makes sense for continuous profiling because part of the appeal here is being able to drill down into particular time ranges.

With adhoc profiling the idea is to be able to take a file as input and then output a flamegraph. Then over time, you can revisit previous ad-hoc profiles that you have created via a file-selector-type-system. This would allow us to recreate all three of our main views:

  • single view
  • comparison view
  • diff view

By simply replacing the timeline input component, with a file-selector input component.

We likely would output the files to a specific folder so this mockup is slightly inaccurate in that sense, but the general idea is there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Mostly go code
Projects
None yet
Development

No branches or pull requests

5 participants