2. Transform the frame

The pipeline now computes the value of each trade, keeps the large ones, and sums them by symbol.

Each step has a schema

pipeline.pymypy: no errors

1class Trades(Columns):

2 symbol: pl.String

3 price: pl.Float64

4 quantity: pl.Int64

5

6

7trades: TypedDataFrame[Trades]

8priced = trades.with_columns(notional=pl.col("price") * pl.col("quantity"))

9large = priced.filter(pl.col("notional") > 10_000)

10by_symbol = large.group_by("symbol").agg(

11 pl.col("notional").sum(),

12 pl.len(),

13)

Resultpriceddeclaredadded

symbolStringpriceFloat64quantityInt64notionalFloat64

Revealed type

TypedDataFrame[TypedDict({'symbol': String, 'price': Float64, 'quantity': Int64, 'notional': Float64}, closed=True), None]

Resultby_symboldeclaredaddeddropped

symbolStringnotionalFloat64lenUInt32pricequantity

Revealed type

TypedDataFrame[TypedDict({'symbol': String, 'notional': Float64, 'len': UInt32}, closed=True), None]

The checker computes the schema of each result:

• with_columns adds notional. A Float64 times an Int64 gives a Float64.
• filter keeps the rows and the columns.
• group_by(...).agg(...) keeps the key and the aggregates. pl.len() counts the rows as a UInt32. The other columns are gone.

A later step that reads price from by_symbol is an error, because the column is gone.

Each step checks its data types

An operation needs columns of the right data types. A mismatch is an error, like a missing column:

pipeline.pymypy: 3 errors

1class Trades(Columns):

2 symbol: pl.String

3 price: pl.Float64

4 quantity: pl.Int64

5

6

7trades: TypedDataFrame[Trades]

8venues = pl.read_csv(

9 "venues.csv",

10 schema={"symbol": pl.Int64, "venue": pl.String},

11)

12trades.filter(pl.col("price"))

↑error: Column "price" is not Boolean, cannot .filter() [wrong-column-shape]

13trades.with_columns(pl.col("symbol") - pl.col("price"))

↑error: "symbol" has no result type at this frame's column types: arithmetic on dtypes String and Float64 is not allowed, and polars raises InvalidOperationError [unsupported-dtype-op]

14trades.join(venues, on="symbol")

↑error: Join key "symbol" is String on the left and Int64 on the right [join-key-dtype]

• filter keeps the rows where a condition is true, so it needs a Boolean column.
• Polars cannot subtract a Float64 from a String.
• A join compares the keys on the two sides, and a String key cannot match an Int64 key.

Next: 3. Write a function that accepts other columns.