Skip to content

Commit 2f7946f

Browse files
authored
PYTHON-4099 Add contributing docs for memory profiling (#2646)
1 parent da6d3d9 commit 2f7946f

File tree

1 file changed

+23
-1
lines changed

1 file changed

+23
-1
lines changed

CONTRIBUTING.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -525,8 +525,10 @@ Use this generated file as a starting point for the completed conversion.
525525

526526
The script is used like so: `python tools/convert_test_to_async.py [test_file.py]`
527527

528-
## Generating a flame graph using py-spy
528+
## CPU profiling
529+
529530
To profile a test script and generate a flame graph, follow these steps:
531+
530532
1. Install `py-spy` if you haven't already:
531533
```bash
532534
pip install py-spy
@@ -536,6 +538,26 @@ To profile a test script and generate a flame graph, follow these steps:
536538
(Note: on macOS you will need to run this command using `sudo` to allow `py-spy` to attach to the Python process.)
537539
4. If you need to include native code (for example the C extensions), profiling should be done on a Linux system, as macOS and Windows do not support the `--native` option of `py-spy`.
538540
Creating an ubuntu Evergreen spawn host and using `scp` to copy the flamegraph `.svg` file back to your local machine is the best way to do this.
541+
5. You can then view the flamegraph using an SVG viewer like a browser.
542+
543+
## Memory profiling
544+
545+
To test for a memory leak or any memory-related issues, the current best tool is [memray](https://bloomberg.github.io/memray/overview.html).
546+
In order to include code from our C extensions, it must be run in native mode, on Linux.
547+
To do so, either spin up an Ubuntu docker container or an Ubuntu Evergreen spawn host.
548+
549+
From the spawn host or Ubuntu image, do the following:
550+
551+
1. Install `memray` if you haven't already:
552+
```bash
553+
pip install memray
554+
```
555+
2. Inside your test script, perform any required setup and then loop over the code you want to profile for improved sampling.
556+
3. Run memray with the script under test with the `--native` flag, e.g. `python -m memray run --native -o test.bin <path/to/script>`.
557+
4. Generate the flamegraph with `python -m memray flamegraph -o test.html test.bin`.
558+
See the [docs](https://bloomberg.github.io/memray/flamegraph.html) for more options.
559+
5. Then, from the host computer, use either scp or docker cp to copy the flamegraph, e.g. `scp [email protected]:/home/ubuntu/test.html .`.
560+
6. You can then view the flamegraph html in a browser.
539561

540562
## Dependabot updates
541563

0 commit comments

Comments
 (0)