ENH: Add SimpleITK Image to ITK Image conversion support

[blowekamp]

This PR introduces a "generic" interface for an "itk-like" python image object. The itk-like object can be converted to a numpy array and contains "spacing", "direction", and "origin" keys. This should be an interface both ITK and SimpleITK images provide.

Any thoughts on this type of interface?

• Implement image_from_simpleitk() function for converting SimpleITK images to ITK images
• Support 2D, 3D, and multi-component (vector) images
• Preserve spacing, origin, and direction metadata
• Preserve MetaData dictionary entries
• Integrate SimpleITK support into filter auto-conversion decorator
• Add comprehensive tests for conversion functionality

PR Checklist

• No API changes were made (or the changes have been approved)
• No major design changes were made (or the changes have been approved)
• Added test (or behavior not changed)
• Updated API documentation (or API not changed)
• Added license to new files (if any)
• Added Python wrapping to new files (if any) as described in ITK Software Guide Section 9.5
• Added ITK examples for all new major features (if any)

Refer to the ITK Software Guide for
further development details if necessary.

[hjmjohnson]

@blowekamp I made a note this morning that this was going to be needed :)! You must have heard my thoughts!

[blowekamp]

@dzenanz Raised this issue here: SimpleITK/SimpleITK#2531

[hjmjohnson]

Can you comment on the missing "Image buffer start index" that is missing?

[thewtex]

👍

[blowekamp]

This feature ran into a issue/block with the ITK dictionary keys unexpectedly return attributes in numpy order and not ITK order.

[hjmjohnson]

I replicated the ordering blocker with released wheels (itk 5.4.6 + SimpleITK 2.5.5): ITK's image['spacing'|'origin'|'direction'] returns numpy (z,y,x) order while SimpleITK's img['spacing'] returns ITK (x,y,z) order — the same string keys with opposite conventions, so a generic reader silently corrupts geometry. The flip is intentional, from d8086fa (2020, @thewtex), implemented at Wrapping/Generators/Python/PyBase/pyBase.i:596-650. Suggested short-term unblock: read geometry via GetSpacing()/GetOrigin()/GetDirection() (identical in both toolkits) instead of dict keys.

Standalone replication script (needs only pip install itk SimpleITK numpy): https://gist.github.com/hjmjohnson/799940ed88d2b9616239ca417f0e9986

Replication results

3-D image, spacing (1, 2, 3), origin (5, 10, 15) in x,y,z:

Interface	['spacing'] returns	Order
itk_image['spacing']	[3. 2. 1.]	numpy (z,y,x) — flipped
sitk_image['spacing']	(1.0, 2.0, 3.0)	ITK (x,y,z)
itk.dict_from_image(img)['spacing']	(1.0, 2.0, 3.0)	ITK (x,y,z)
GetSpacing() (both toolkits)	(1.0, 2.0, 3.0)	ITK (x,y,z)

So ITK is also internally inconsistent: dict(image)['spacing'] ≠ itk.dict_from_image(image)['spacing'].

The 'direction' key is the worst case: np.flip(matrix, axis=None) on a 30° rotation matrix yields its transpose (the inverse rotation) — silently wrong geometry rather than an obviously reversed tuple.

Feeding this PR's generic-reader pattern (read src['spacing'], call dst.SetSpacing()) an ITK-convention object corrupts spacing (1,2,3) → (3,2,1).

Separate finding: against released SimpleITK 2.5.5, np.array(sitk_image) falls back to pixel iteration (flat 1-D float64 array), so image_from_simpleitk raises TemplateTypeError; the PR effectively requires SimpleITK 3.x-dev's shaped __array__ (SimpleITK/SimpleITK#2531) and should probably gate on that or use sitk.GetArrayViewFromImage.

Possible paths forward

1. Unblock this PR without a convention change: use GetSpacing()/GetOrigin()/GetDirection() for geometry and GetArrayViewFromImage for pixels.
2. Fix the convention at the source: change ITK's __getitem__/__setitem__/keys() spatial keys to ITK (x,y,z) order with a deprecation period. This also resolves the dict(image) vs dict_from_image inconsistency. Since numpy order was a deliberate 2020 design choice (consistency with np.array(image) indexing), @thewtex should weigh in.
3. Coordinate the shared key convention with SimpleITK before 3.x cements its side.

[greptile-apps]

Greptile Summary

This PR adds SimpleITK image support to ITK's Python conversion helpers. The main changes are:

• New itk.image_from_simpleitk() conversion helper.
• SimpleITK input handling in the Python filter auto-conversion decorator.
• Tests for scalar images, vector images, spatial metadata, metadata dictionaries, and filter inputs.

Confidence Score: 4/5

One contained bug needs to be fixed before merging.

The main change is localized, but current SimpleITK images do not provide spacing, origin, or direction through the generic key lookup used here, so required metadata preservation and vector dimensionality can be wrong.

Wrapping/Generators/Python/itk/support/extras.py

T-Rex Logs

What T-Rex did

• Ran a real SimpleITK repro script against the repository's image_from_simpleitk path and confirmed SimpleITK getters report spacing, origin, and direction, but the conversion did not preserve a 3-component vector image (one component after conversion).
• Generated a confirming proof for a posted P1 finding, aligning with the reviewer’s P1 details.
• Compared post-change logs and confirmed an import failure: ModuleNotFoundError: No module named 'itk.Configuration' when importing the repo-local itk.
• Analyzed the repo function path and found that loading from the repo and converting a real sitk.Float32 2D image failed because numpy.array(sitk_image) produced a one-dimensional object array, causing ITK to request an unsupported image type.
• Verified pre-syntax checks by running py_compile on Tests/extras.py and itk/support helpers, which passed.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
Wrapping/Generators/Python/Tests/extras.py	Adds SimpleITK conversion coverage for scalar, vector, metadata, and filter auto-conversion paths.
Wrapping/Generators/Python/itk/support/extras.py	Adds image_from_simpleitk, but real SimpleITK spatial metadata is not read through the implemented generic key fallback.
Wrapping/Generators/Python/itk/support/helpers.py	Extends the array-like conversion decorator to accept SimpleITK images and return ITK outputs without reconversion.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Caller
participant Decorator as accept_array_like_xarray_torch
participant Converter as itk.image_from_simpleitk
participant Filter as ITK filter
Caller->>Decorator: filter(SimpleITK.Image)
Decorator->>Converter: convert input image
Converter-->>Decorator: itk.Image / itk.VectorImage
Decorator->>Filter: invoke with ITK image
Filter-->>Decorator: ITK output
Decorator-->>Caller: ITK output

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Caller
participant Decorator as accept_array_like_xarray_torch
participant Converter as itk.image_from_simpleitk
participant Filter as ITK filter
Caller->>Decorator: filter(SimpleITK.Image)
Decorator->>Converter: convert input image
Converter-->>Decorator: itk.Image / itk.VectorImage
Decorator->>Filter: invoke with ITK image
Filter-->>Decorator: ITK output
Decorator-->>Caller: ITK output

Loading

Comments Outside Diff (1)

1. General comment

   image_from_simpleitk does not obtain pixel arrays from real SimpleITK images

   • Bug
     • The generated PR test-plan harness used a real SimpleITK.Image([12, 8], sitk.sitkFloat32) and executed the changed repo implementation of image_from_simpleitk(). It failed immediately during itk.image_view_from_array(array), before metadata or filter auto-conversion could be validated. The stack trace shows ITK attempted itk.Image[itk.D, 1], which is consistent with np.array(sitk_image) not returning the expected 2D float pixel buffer for SimpleITK images.
   • Cause
     • image_from_simpleitk() reads pixels with array = np.array(sitk_image). Real SimpleITK images do not expose their pixel buffer through NumPy's generic array protocol in the required shape/dtype; SimpleITK's supported API is SimpleITK.GetArrayFromImage() / GetArrayViewFromImage(). As a result, scalar 2D images are misinterpreted as a 1D double/object-like array and conversion fails.
   • Fix
     • Use SimpleITK's image array API when the input is a SimpleITK image, e.g. sitk.GetArrayFromImage(sitk_image), while preserving the intended vector-image channel layout. Keep the generic path only for objects that actually provide a valid ndarray via __array__, and add a runtime test that checks the array shape/dtype before calling itk.image_view_from_array().

   _{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "ENH: Add SimpleITK Image to ITK Image co..." | Re-trigger Greptile}

[greptile-apps]

Use SimpleITK getters
The fallback path never reads GetSpacing(), GetOrigin(), or GetDirection() from real SimpleITK.Image objects, because SimpleITK exposes those as methods rather than image['spacing'] / keys(). With only GetMetaDataKeys() added to keys, spacing is absent, so dim stays as array.ndim, vector images are treated as scalar 3D images, and spatial metadata is left at ITK defaults; the new spacing/origin/direction tests fail against current SimpleITK.

Context Used: AGENTS.md (source)

Artifacts

Repro: Python script creating real SimpleITK scalar and vector images and checking converted ITK metadata and component count

• Contains supporting evidence from the run (text/x-python; charset=utf-8).

Repro: failing execution log with package versions, SimpleITK getter values, converted ITK properties, mismatches, and assertion failure

• Keeps the command output available without making the summary code-heavy.

_{Ran code and verified through T-Rex}

[thewtex]

consistency with np.array(image) indexing

I think this is still important.

[dzenanz]

It would be good if this was finished before the next tag.

[hjmjohnson]

It would be good if this was finished before the next tag.

If feels like we might need a short meeting to discuss the pros/cons of the various possible solutions, and once the solution is picked, it will be very easy to implement. My reading is we are stuck at a point of two conflicting design decisions (support numpy native, support ITK/simpleITK native conventions). Perhaps we need to break both and have a new naming scheme.

"numpy_spacing" = reverse ("itk_spacing")

[hjmjohnson]

@blowekamp — thank you for this, and apologies it sat so long. The conversion logic here is the basis of what finally landed as a follow-on, and you are credited as co-author on the commit.

The blocker was never this PR's approach; it was that the underlying dict protocol was ambiguous. image['spacing'] meant (z, y, x) on an itk.Image and (x, y, z) on a SimpleITK Image, so any converter reading it silently reversed the spacing (#6706). That is now fixed upstream of the converter: #6710 added order-explicit spacing_xyz / origin_xyz / direction_xyz keys, and #6811 rebuilds this work on top of them.

#6811 carries your implementation forward with the review concerns on this PR addressed:

• greptile's P1 — geometry now falls back to GetSpacing() / GetOrigin() / GetDirection(), since SimpleITK exposes methods rather than dict keys, and component count comes from GetNumberOfComponentsPerPixel() so vector images are no longer treated as scalar.
• The buffer start index — a non-zero buffered-region index is carried as ITK_original_index metadata and restored by the inverse. Your point that SimpleITK enforces a zero start index holds for sitk→itk; ENH: Add SimpleITK <-> ITK image conversion to the Python interface (supersedes #6021) #6811 adds the itk→sitk direction, where the index does need somewhere to live.

Also worth flagging for your question on SimpleITK/SimpleITK#2531 ("is there a generic way we can support conversion between ITK Python and SimpleITK — can both implementations accept an array buffer and these keys?"): ITK's half is answered, but SimpleITK exposes no order-explicit keys as of 3.0.0b1, so the accessor fallback carries every conversion today. If SimpleITK adds them, the key path starts working with no code change.

Verified end to end against SimpleITK 3.0.0b1 built with SimpleITK_USE_SYSTEM_ITK=ON against the same ITK 6 that produced the Python bindings, so both toolkits share one ITK.

Suggest closing this in favor of #6811, but leaving that to you.