0.1.18.beta4

What's Changed

• Adding a data model to represent compact_partition function parameters by @pfaraone in #151
• [skip download cold manifest] Add support for skipping download cold manifest entries during materialize step by @Zyiqin-Miranda in #147
• Capturing all the performance metrics in an audit by @raghumdani in #146

Full Changelog: 0.1.18.beta3...0.1.18.beta4

0.1.18.beta3

What's Changed

• Added new compute step 'repartition', in deltacat, to support different repartition strategy, e.g., hash, range, column by @valiantljk in #126
• Repartition unit tests by @valiantljk in #136
• Enabling skipping untouched files during materialize by @Zyiqin-Miranda in #137
• Setting hash bucket count based on POC results by @raghumdani in #139
• Logging memory consumed to validate worker estimation correctness by @raghumdani in #142
• Removing dependency on rebase_source_partition by @valiantljk in #143

Full Changelog: 0.1.18.beta1...0.1.18.beta3

0.1.18.beta2

Update to 0.1.18

0.1.18.dev

What's Changed

• Add assertions to fast fail compaction in case of a metadata/execution issue by @rkenmi in #126

Full Changelog: 0.1.17...0.1.18.dev

0.1.18.beta1

What's Changed

• Update hash bucket and dedupe result dataclasses to tuple type by @rkenmi in #129

Full Changelog: 0.1.16...0.1.18.beta1

0.1.16

What's Changed

• Add an arg for list_deltas kwargs to disable compacted table resolution by @raghumdani in #110
• adding kwargs to all invocations of _discover_deltas by @raghumdani in #111
• fixing high watermark deserialization and TypeError by @raghumdani in #112
• raise error when the compaction requires multiple rounds by @raghumdani in #114

Full Changelog: 0.1.15...0.1.16

0.1.15

What's Changed

• [profiler] Add memray profiler to compaction by @Zyiqin-Miranda in #88
• Fix compactor code path with only source partition to compact from and no previous rounds by @rkenmi in #99
• [ci][linting] Updated pre-commit repository mapping to be compatible with Python 3.7 by @pfaraone in #98
• [metrics] Add customized CloudWatch metrics support by @Zyiqin-Miranda in #96
• Make memray install optional and support custom logging config by @pdames in #103
• Compaction with no PKI by @valiantljk in #86
• deltacat version bump and support for graviton by @raghumdani in #104
• add property info by @valiantljk in #102
• Fix key type by @valiantljk in #106

Full Changelog: 0.1.12...0.1.15

0.1.12

What's Changed

• Hotfix: Remove @ray.remote annotation in compaction session and zero_copy_only in materialize by @rkenmi in #95

Full Changelog: 0.1.11...0.1.12

0.1.11

What's Changed

• Support table compaction source rebases and refactor compact_partition by @pdames in #70
• Add option to coerce int96 timestamp precision during Parquet reads by @rkenmi in #74
• Fix dedupe when iterchunks() doesn't return equal length chunks across columns by @raghumdani in #78
• Enhance logging observability with Ray Runtime Context and Process ID by @rkenmi in #85
• Maximize file sizes produced by each materialize task by @rkenmi in #71
• #37- Retroactively applying linter (black, isort) + Lint on CI by @pfaraone in #47

Full Changelog: 0.1.10...0.1.11

0.1.10.dev

bumped version