Skip to main content

How to Build a Python-to-C++ Compiler out of Spare Parts - and Why

Track:
Python Internals & Ecosystem
Type:
Talk
Level:
advanced
Room:
Forum Hall
Start:
14:35 on 11 July 2024
Duration:
30 minutes

Abstract

A frequent topic about Python is performance: its interpreted nature inhibits optimisations, and the famous GIL limits parallelism (for now!).

Existing Python Compilers - Cython, Numba, Codon - focus mainly on compiling small, critical bits of code to achieve linear execution speedups. As for parallelism: parallel for-loops powered by OpenMP.

To parallelize highly concurrent programs with concurrent I/O and concurrent tasks, we need more. A key difference is it requires compiling everything: as soon as the Python interpreter comes into play, the GIL will make parallelism collapse.

We introduce Typon, a Python-to-C++ compiler with powerful concurrency primitives powered by a crazy homemade task scheduler. It can take untyped, idiomatic Python code and output C++ code fully independent of the Python interpreter. It also provides seamless to-and-from Python interoperability, for those cases where you really just need to import numpy.

In this talk we’ll recount our journey so far: why we think it’s important, how we’re making something new out of existing bits, what we’ve achieved. Along the way we might delve into fun details like type inference, concurrency primitives, and C++ pretending-to-be-Python.

You’ll come out of this talk with some cool insights into compiler design, concurrency, and the design of Python.

Knowledge of C++ not required. Knowledge of Python language inner workings helpful.


The speakers

Tom Niget

Tom Niget

I make things with computers. I enjoy solving problems with code, and building interesting software to play with data and algorithms.

Xavier Thompson

Xavier Thompson

Xavier embarked on what would become the Typon project as an intern at Nexedi and hasn’t looked back since. When he’s not thinking about compiler design and type systems, he enjoys swing dancing and singing in a choir. This will be his second time talking at a conference.