Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/freshdata/adapters/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
"""Optional framework adapters (Polars, etc.)."""
"""Optional framework adapters (Polars, etc.).

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from .polars import from_pandas, is_polars_frame, to_pandas

Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/compliance/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
per-column roles and missing ratios via :func:`freshdata.infer_roles`) and/or an
``enterprise_result=`` (to fold in the 0–100 Data Trust Score, PII-masking
events, and fuzzy-clustering lineage) when richer evidence is available.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/context/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
:class:`ContextPolicy` that lowers into the existing :class:`~freshdata.CleanConfig`
machinery. Fully offline, model-free, and dependency-free — unresolved or
unparsed sentences are always surfaced, never guessed at.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from .compiler import apply_policy_to_config, compile_context, resolve_policy
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/domains/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@

Third-party packs register via the ``freshdata.domains`` entry-point group; see
``CONTRIBUTING_DOMAINS.md``.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/engine/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
dataset (size, duplicate ratio), then chooses cleaning actions from explicit
threshold rules. Every decision — including the decision to leave a column
untouched — is logged with a rationale, a risk level, and a confidence score.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from .context import ColumnContext, build_contexts, infer_role
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/enterprise/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
Optional dependencies are imported lazily, so ``import freshdata`` stays cheap and
pandas-only installs keep working; the Polars-native fast paths activate automatically when
polars is installed.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from .cleaner import (
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/execution/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

Public entry point: :func:`run_with_engine`, wired into :func:`freshdata.clean`
via its ``engine`` / ``output_format`` / ``engine_config`` keyword arguments.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/experimental/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
:mod:`freshdata.experimental.ai_copilot`
Deterministic, privacy-first dataset analysis that produces an
explainable cleaning plan and copy-ready freshdata code.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
6 changes: 5 additions & 1 deletion src/freshdata/imputation/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
"""Internal imputation engines."""
"""Internal imputation engines.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

__all__: list[str] = []
3 changes: 3 additions & 0 deletions src/freshdata/integrations/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@

Install the extras as needed, e.g. ``pip install "freshdata-cleaner[dagster]"`` or
``pip install "freshdata-cleaner[integrations]"`` for all three.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/learning/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
examples, and an embedded :class:`~freshdata.CleaningMemory` — into an
auditable profile that replays through the exact same policy gates as every
other proposal source.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
fd.models.pull("fd-col-encoder-v1") # explicit download (network)
fd.models.path("fd-col-encoder-v1") # local artifact path
fd.models.list_available() # registry metadata

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from .download import pull
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/parsers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
:class:`ParseResult`; the frames can then be cleaned and domain-validated with
:func:`freshdata.clean`. See :func:`freshdata.parse_domain` and
:func:`freshdata.clean_domain_file` for the high-level entry points.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/render/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
vanilla JS for filtering/collapsing) with *zero* optional dependencies. The
``freshdata-cleaner[viz]`` / ``freshdata-cleaner[notebook]`` extras (itables, plotly,
great-tables, anywidget) merely *upgrade* the output when installed.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/semantic/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
- ``policy`` is the single place to add semantic-memory replay and privacy
redaction (``semantic_privacy_policy``) before any future external inference.
- proposals project cleanly to OpenLineage / quality-ops events.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
3 changes: 3 additions & 0 deletions src/freshdata/steps/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,7 @@
Steps never mutate the caller's original DataFrame: the pipeline hands them a
frame it owns, and steps only ever rebind whole columns or produce new frames
(``.loc`` row selection), never write into shared blocks in place.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""
3 changes: 3 additions & 0 deletions src/freshdata/streaming/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
The cleaner keeps **bounded** running statistics across batches (Welford mean/variance,
reservoir-sampled medians, Space-Saving top-k categories), so memory stays flat whether
you feed it 100k rows or 100M. See :class:`StreamingCleaner`.

See `ARCHITECTURE.md <https://github.com/FreshCode-Org/freshdata/blob/main/ARCHITECTURE.md>`_
for how this package fits into the overall cleaning flow.
"""

from __future__ import annotations
Expand Down
Loading