Making PyPI

Summary: Trail of Bits cut the Warehouse test suite that powers PyPI from 163 seconds to 30 seconds, an 81% improvement, while the test count grew from 3,900 to over 4,700. The gains came from parallelizing tests with pytest-xdist, switching coverage to the sys.monitoring API, restricting test discovery with testpaths, trimming unused imports, and a migration-squashing proof of concept that maintainers declined to merge.
Key Takeaway: PyPI serves over one billion distribution downloads per day, and its 4,734 tests enforce 100% branch coverage on every pull request; the maintainers rejected a further 13% speedup because the added migration complexity outweighed the gain, showing that not every metric improvement is worth the long-term maintenance cost.

Read the original at Trail of Bits Blog ↗