Skip to content

Guides

Open frames

A function can take any frame that has some columns. The caller gets the other columns back, with their names and their data types.

An open frame has declared columns and a remainder. The remainder stands for the columns that the type does not name. Declare it with other_columns(), and write it as the second argument of TypedDataFrame.

Below, add_notional accepts any frame with the columns of Trades, and Rest stands for the other columns. At each call, the checker puts the columns of the caller in place of Rest. A Blotter frame has one more column, venue, and the result keeps it:

notional.pymypy: no errors
1class Trades(Columns):
2 symbol: pl.String
3 price: pl.Float64
4 quantity: pl.Int64
5
6
7class WithNotional(Trades):
8 notional: pl.Float64
9
10
11Rest = other_columns()
12
13
14def add_notional(
15 df: TypedDataFrame[Trades, Rest],
16) -> TypedDataFrame[WithNotional, InferredRemainder]:
17 return infer(df.with_columns(notional=pl.col("price") * pl.col("quantity")))
18
19
20class Blotter(Trades):
21 venue: pl.String
22
23
24blotter: TypedDataFrame[Blotter]
25result = add_notional(blotter)
Resultresultdeclaredaddedfrom the remainder
symbolStringpriceFloat64quantityInt64notionalFloat64venueString
Revealed type
TypedDataFrame[TypedDict({'symbol': String, 'price': Float64, 'quantity': Int64, 'notional': Float64, 'venue': String}, closed=True), None]

The return type writes InferredRemainder for the remainder of the result, and the body wraps the returned value in infer(...). The checker then computes the remainder from the body. If a caller passes a notional column, with_columns replaces it, so the remainder of the result never holds notional. In a revealed type inside the function, the checker writes that remainder as Minus[Rest, Literal["notional"]]. See Inferred return types and Remainders.

In the body, the checker knows only the declared columns. A caller can pass a frame without venue, so a read of venue is an error:

venues.pymypy: 1 error
1class Trades(Columns):
2 symbol: pl.String
3 price: pl.Float64
4 quantity: pl.Int64
5
6
7Rest = other_columns()
8
9
10def by_venue(df: TypedDataFrame[Trades, Rest]) -> None:
11 df.select("symbol", "venue")
error: Column "venue" is not declared and cannot be proven to exist [undeclared-column]
note: set allow_unproven = "possible" in [tool.astrolabe] to allow this, or set [tool.astrolabe.allow_unproven_overrides] select = "possible"

A keyword of other_columns(...) states a promise that each caller must keep. The body can rely on it, and the checker reports a call that breaks it:

notional.pymypy: 1 error
1class Trades(Columns):
2 symbol: pl.String
3 price: pl.Float64
4 quantity: pl.Int64
5
6
7Clean = other_columns(without=["notional"])
8
9
10def add_notional(df: TypedDataFrame[Trades, Clean]) -> TypedDataFrame[Trades, Clean]:
11 return df
12
13
14class Booked(Trades):
15 notional: pl.Float64
16
17
18booked: TypedDataFrame[Booked]
19add_notional(booked)
error: Remainder "Clean" must not contain column "notional" [forbidden-column]
KeywordThe promise
without=[...]The remainder holds none of these names.
only=[...]The remainder holds only these names.
without_prefix=, only_prefix=No name starts with the prefix, or every name does.
without_suffix=, only_suffix=No name ends with the suffix, or every name does.
all_of=Every column has one of these data types.
none_of=No column has one of these data types.
disjoint_from=This remainder shares no name with the other remainders.
within=The other remainders hold every name of this one.

all_of= and none_of= take a Polars class such as NumericType or pl.String, a data type with parameters such as pl.Datetime("us"), or a set of them. AnyZone fills the zone of a Datetime with every zone of its time unit: Datetime[Literal["us"], AnyZone] is the data types that cs.datetime("us") picks.

A join keeps the columns of both frames. If both remainders can hold one name, Polars renames the right column for some callers and not for others. The checker cannot name the columns of such a result:

enrich.pymypy: 1 error
1class Keys(Columns):
2 symbol: pl.String
3
4
5Left = other_columns()
6Right = other_columns()
7
8
9def enrich(left: TypedDataFrame[Keys, Left], right: TypedDataFrame[Keys, Right]) -> None:
10 left.join(right, on="symbol")
error: This join renames columns for some callers but not others: the two remainders can share a name. Declare disjoint_from= on one of them [join-rename]
note: set allow_unproven = "possible" in [tool.astrolabe] to allow this, or set [tool.astrolabe.allow_unproven_overrides] join = "possible"

disjoint_from= states that the two remainders share no name. The result then holds the columns of both remainders. At the call below, bid comes from the left frame and venue from the right:

enrich.pymypy: no errors
1import polars as pl
2
3from astrolabe import Columns, InferredRemainder, TypedDataFrame, infer, other_columns
4
5
6class Keys(Columns):
7 symbol: pl.String
8
9
10Left = other_columns()
11Right = other_columns(disjoint_from=Left)
12
13
14def enrich(
15 left: TypedDataFrame[Keys, Left],
16 right: TypedDataFrame[Keys, Right],
17) -> TypedDataFrame[Keys, InferredRemainder]:
18 return infer(left.join(right, on="symbol"))
19
20
21class Quotes(Keys):
22 bid: pl.Float64
23
24
25class Venues(Keys):
26 venue: pl.String
27
28
29quotes: TypedDataFrame[Quotes]
30venues: TypedDataFrame[Venues]
Resultenrich(quotes, venues)declaredfrom the remainder
symbolStringbidFloat64venueString
Revealed type
TypedDataFrame[TypedDict({'symbol': String, 'bid': Float64, 'venue': String}, closed=True), None]