Skip to content

Optimize interface a python basic - #843

Merged
dmitry-kabanov merged 58 commits into
MaRDI4NFDI:mainfrom
dmitry-kabanov:optimize-interface-a-python-basic
Jan 28, 2026
Merged

Optimize interface a python basic#843
dmitry-kabanov merged 58 commits into
MaRDI4NFDI:mainfrom
dmitry-kabanov:optimize-interface-a-python-basic

Conversation

@dmitry-kabanov

@dmitry-kabanov dmitry-kabanov commented Jan 12, 2026

Copy link
Copy Markdown
Collaborator
  • Add a basic version of the Optim interface and a Python implementation using scipy.optimize.
  • Add an ability to return values from implementations, because Python cannot mutate strings and we need it to return a message about optimization results.
  • Add tests

@dmitry-kabanov
dmitry-kabanov force-pushed the optimize-interface-a-python-basic branch from b7192db to f6b8df7 Compare January 27, 2026 15:36
@dmitry-kabanov
[python, optim] Work on optimization interface
2d5a317
@dmitry-kabanov
[python, optim, tests] Add initial tests
5a2abff
@dmitry-kabanov
[docs] Add technote on development of optimization interface
a654fb7
@dmitry-kabanov
[docs] Analyze interfaces in JuMP and Scipy Optimize
47f5b82
@dmitry-kabanov
[docs] Add missing files for version bump to technote
065e098
@dmitry-kabanov
[python, optim] Remove non-mutable arguments
2631ef5
@dmitry-kabanov
[misc] Update clang-tidy config to avoid warnings about Python C API …
a8f5622 
…usage
@dmitry-kabanov
[python, bridge] Convert return arguments to C representation
970d83c
@dmitry-kabanov
[c, julia] Handle return arguments
29bd26b
@dmitry-kabanov
[python, bridge] Accept return value and check that the number of exp…
93911d9 
…ected arguments matches
@dmitry-kabanov
[python, converter] Pass return_args argument as a pointer as it will…
7f2f703 
… be modified
@dmitry-kabanov
[python, converter] Release memory of return args after copying them
3e42dd9
@dmitry-kabanov
[python, bridge] Correct convert Python string object to C null-termi…
3c78a2d 
…nated string
@dmitry-kabanov
[python, converter] Work on convert return values to proper Python va…
48e5091 
…riables
@dmitry-kabanov
[python, bridge] Fix memory leak when converting Python string to C s…
d5a1004 
…tring
@dmitry-kabanov
[python] Correctly convert return Python objects to C intermediate re…
118084b 
…presentation
@dmitry-kabanov
[python, converter] Correctly convert C intermediate return args to P…
189a3fc 
…ython objects
@dmitry-kabanov
[scipy_optimize] Return arguments from minimize function
2966e52
@dmitry-kabanov
[tests, python] Update tests for optim interface
57c8fc7
@dmitry-kabanov
[python, optim] Implement set_user_data method
f9c571b
@dmitry-kabanov
[python, Optim] Clean code - docstrings, redundant code
331fda1
@dmitry-kabanov
[tests, python] Add test for optim::set_user_data function
71a00d3
@dmitry-kabanov
[python] Implement method optim::set_method in scipy_optimize
729d9a6
@dmitry-kabanov
[python] Add method optim::set_method to the Optim gateway
7c1d339
@dmitry-kabanov
[docs] Update documentation
c2cfdd7
@dmitry-kabanov
[python, tests] Add tests for optimization results getting better wit…
df12ab5 
…h stricter tolerances
@dmitry-kabanov
[examples] Update optimization example for Python
2c364a1
@dmitry-kabanov
[python, misc] Fix wrong constant name
c47eb46
@dmitry-kabanov
[python, bridge] Allocate memory for a return integer only if all che…
d4d1ddd 
…cks have passed
@dmitry-kabanov
[python, optim::set_method] Check that passed method parameters are a…
e01cae5 
…ccepted
@dmitry-kabanov
[python, scipy_optimize] Type hint properties correctly
9a537b0
@dmitry-kabanov
[python, scipy_optimize] Check available method options correctly
691cd3c
@dmitry-kabanov
[python, scipy_optimize] Check preconditions for minimize
c2fc064
@dmitry-kabanov
[python, converter] Raise exception on the error before converting re…
e583120 
…turn arguments
@dmitry-kabanov
[python, Optim gateway] Hold reference to user_data
960a1e2
@dmitry-kabanov
[python, converter] Add a reminder to make_oif_user_data docstring …
50c0a06 
…to keep reference
@dmitry-kabanov
[python, Bridge] Initialize Python objects to NULL
d14f6d6
@dmitry-kabanov
[python, Bridge] Load libpython and NumPy only once
95a797c 
Somehow, this small refactoring took too much time, and I still do not quite understand why.
For some reason, the reference to the sysconfig module must be preserved.
@dmitry-kabanov
[Julia, Converter] Handle situation when no return args are passed fo…
baa1632 
…r backward compatibility
@dmitry-kabanov
[python, Optim gateway] Keep reference to x0 to extend its lifetime
a770680
@dmitry-kabanov
[python, Bridge] Fix C errors (uninitialized pointers, wrong type for…
790dc6b 
… strides)
@dmitry-kabanov
[python, converter] Use keyword arguments to create an OIFUserData st…
b29fe8c 
…ruct
@dmitry-kabanov
[python, Bridge] Use cached deserialization function instead of impor…
54dec4b 
…ting it each time
@dmitry-kabanov
[python, Bridge] Extract a function that parses impl_details
bf206c3
@dmitry-kabanov
[python, bridge] Fix a couple of memory leaks and redundant memory ma…
78462c5 
…nagement
@dmitry-kabanov
[python, bridge] Remove unused logic for output argument types that c…
2b2a88e 
…annot be mutable
@dmitry-kabanov
[python, bridge] Fix a reference deficit with a Python OIF_USER_DATA …
1bcce68 
…object

For some reason, this problem has never manifested itself before,
although it is clear from the code that I have the same fix
for a Python OIF_CALLBACK object.
When I have started to work on the `optim` interface, this reference deficit
started to happen quite often, hence the fix with `Py_INCREF`.
Reference deficit is when a `PyObject *` does not have its refcount
incremented enough times.
@dmitry-kabanov
[c] Copy dimensions array in oif_create_array_f64 to avoid dangling p…
8af61cf 
…ointers

If the user passes to `oif_create_array_f64` for the `dimensions` parameter
a stack-allocated array, then it can easily happen that in the created array
the field `dimensions` will be a dangling pointer at some point.
To prevent this, we copy the passed array so that it stays with the array
during its lifetime.
@dmitry-kabanov
[c] Fix a bug in calculation of the number of required bytes for `dim…
3620a6d 
…ensions` field
@dmitry-kabanov
[c] Check for overflow when computing the size of the array's data
197799a 
[misc] Debug CI
@dmitry-kabanov
[c, julia, python] Fix a bug with mismatched types for OIFArrayF64->d…
0911ed5 
…imensions

I really do not know how this thing were working before:
Python was using 32-bit integers while C and Julia 64-bit integers to represent
dimensions of an array.
How they were able to convert them, remains unknown.
@dmitry-kabanov
[python, bridge] Fix memory leaks and double decref's
090d10d
@dmitry-kabanov
[misc] Fix pre-commit issues
1009373
@dmitry-kabanov
dmitry-kabanov force-pushed the optimize-interface-a-python-basic branch from f6b8df7 to 1009373 Compare January 27, 2026 16:19
@dmitry-kabanov
dmitry-kabanov force-pushed the optimize-interface-a-python-basic branch from 2c5ad6f to 413e47b Compare January 28, 2026 17:39
@dmitry-kabanov
dmitry-kabanov merged commit cfe7623 into MaRDI4NFDI:main Jan 28, 2026
7 checks passed
@dmitry-kabanov
dmitry-kabanov deleted the optimize-interface-a-python-basic branch April 15, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dmitry-kabanov