Changelog

v0.4.0 (2026-09-12)

Filtering multiple files without --columns failed when a column was null in one file and int32 in another. It now combines compatible column types automatically and retains all columns.
Numeric types depended on the first file. For example, uint8 followed by double could fail on fractional values; both orders now use double.
A column present only in a later file was omitted from output and could not be filtered. It is now available in either file order, with nulls where absent.
Stored pandas dtypes could narrow promoted values, and a missing earlier column could round 2**53 + 1. Nullable integer widening and newly inserted nulls now preserve 64-bit integer values in pqfilt.read().
Increase the minimum PyArrow version from 10 to 14 for numeric schema promotion.

Save output atomically: failures preserve existing files; overwrite=False also protects files created during the scan.
Reject hard links to input files as streaming output destinations.
Expand glob patterns in source lists and accept literal filenames containing brackets. Read each file once, including repeated paths and file aliases.
Preserve Arrow’s unknown result for DataFrame comparisons against None, including negated comparisons.
Validate release versions and installed distributions before publishing. Select CI Python versions explicitly and exclude local planning files from sdists.

Critical bug fix: * Column names ending in in were misinterpreted as the in operator. Fixed. * Fixed parsing of parenthesized membership lists * Fixed handling of unterminated quotes.
Made pqfilt.filter_df() match Arrow null behavior
pqfilt.filter_df(): DNF tuple validation consistent and explicit.
Added pqfilt.scan(): Arrow-native, lazily evaluated scans
Added pqfilt.write_filtered(): One-shot writer (csv or parquet) t * The CLI now uses this streaming write path.
Added public AST utilities (pqfilt.to_ast(), pqfilt.map_leaves()) and operator utilities (pqfilt.SUPPORTED_OPERATORS, pqfilt.validate_operator()).
Preserved exact integer filter literals beyond the binary64 precision limit.
Split documentation into a practical usage guide and an advanced API guide.

Critical bug in CLI: per_file option was removed in 0.2.0 but CLI still referenced it, causing a crash. Fixed.

Breaking: removed per_file parameter from read().
Added is null / is not null operators.
Added ~ prefix negation (NotExpr).
Fixed quoted literals ('a|b', 'a&b', etc.) being split on structural characters.
Fixed in/not in list items with commas inside quotes (desig in 'a,b', 'c').
Fixed boolean column filtering: True/False strings now coerce to bool before pyarrow pushdown.
Multi-file reads now use a single ds.dataset(files) call (parallel I/O).
Parquet output now uses pq.write_table to preserve Arrow type fidelity.
Added filter_df(df, filters) to apply the same filter syntax to an already-loaded DataFrame.

True/False string literals (case-insensitive) are now converted to Python bool before pyarrow, preventing type-mismatch errors on boolean columns.

list/tuple: in/not in value lists now accept bracket/parenthesis enclosures. * "desig in [1, 2, 3]" and "desig in (1, 2, 3)" work in addition to the bare comma-separated form.
Bugfix a few edge cases.

Similar fix as 0.1.3: Fixed in/not in value parsing * "desig in '356', '3200'" are preserved as strings.

Fixed quoted scalar values * "name == '42'" now correctly preserves '42' as a string instead of coercing it to the integer 42.