Robotic python - #215
Draft
JorisGoosen wants to merge 16 commits into
Draft
Conversation
added 16 commits
August 17, 2026 18:28
Adds tests/equivalence/: dumps the full RCPP_MODULE surface and three golden JSONs (response, post-changeOptions prune, saved convertToJSON tree) from a rich results object. runGate.sh rebuilds the working tree and byte-compares. Fixtures generated from the pre-refactor build; must stay identical through the core/adapters split.
New R-free src/core/ starts with jaspHost: the callback/store seam that will replace everything currently hardwired to R (log, send, poll, abort, column-name decode, plot/state object store, plot rendering, state archive). Defaults are neutral no-ops; not referenced by anything yet. configure(.win) compile the new core/*.cpp sources. Equivalence gate + testthat green.
…rsions/interface/toRObject jaspObject and jaspEnums move to src/core/ R-free: std::string/Json::Value signatures, logging + column-name decoding through jaspHost, dependency API on Json::Value. The SEXP->Json conversion templates move to adapters/rcpp/rcppConversions (free functions, per-object _escapeHtml passed explicitly), jaspObject_Interface + module macros to adapters/rcpp/jaspObjectInterface, and the former virtual toRObject() becomes rcppToRObject() type dispatch. jaspContainer::setError takes std::string. configure(.win) compile adapters/rcpp/*.cpp and add -I flags. Equivalence gate (fingerprint + 3 goldens) and testthat green.
jaspHtml and jaspQmlSource move to src/core/ R-free (std::string ctor, setValue takes Json::Value; the unused Rcpp-only RcppVector_to_ArrayJson helper is dropped). Their _Interface wrappers move to the new adapters/rcpp/rcppInterfaces.h (grows as more classes move) and jaspHtml's toRObject becomes a static builder inside rcppToRObject.cpp. jaspReport and jaspState stay for now: they depend on jaspResults (analysisId / object store) and move with it. Equivalence gate (fingerprint + 3 goldens) and testthat green.
analysisId (which analysis is running) becomes jaspHost state instead of a private jaspResults static: jaspResults::setResponseData/analysisId now delegate to jaspHost, and jaspReport/jaspColumn read jaspHost::analysisId(). This unblocks core classes that need the id without the jaspResults object. The jaspHost object store also becomes overridable std::function callbacks (default in-process map) so the R engine can back it with a GC-protected environment. jaspReport moves to src/core/ R-free (std::string ctor/setText); its _Interface moves to rcppInterfaces.h. Equivalence gate (fingerprint + 3 goldens) and testthat green.
Adds toRObjectBaseline.R, a structural fingerprint of the toRObject() tree (S3 classes, names, data.frame column types/titles). The JSON goldens never covered the R-side object conversion, which commit 03 rewired from virtual dispatch to rcppToRObject() type dispatch; this fixture pins it before jaspContainer and jaspTable move. runGate.sh compares it alongside the other fixtures.
Covers the path JSON goldens cannot: jaspState objects stored in the _RStorageEnv, retrieved via getObject, and harvested by getOtherObjectsForState/getKeepList. Baseline captured pre-migration so the jaspHost store wiring can be verified against it.
jaspState moves to src/core/ R-free: setObject/getObject carry opaque std::any through jaspHost's object store instead of calling jaspResults' Rcpp env directly. rcppWireHostStore() (called from the jaspResults constructor) points the jaspHost store callbacks at jaspResults::_RStorageEnv, so R objects stay GC-protected exactly as before; jaspState_Interface in rcppInterfaces.h keeps the R-facing SEXP API. jaspResults' state harvest now reads through the seam. The state-store round-trip + harvest fixture (commit 07a) is byte-identical, confirming the migration preserves R behaviour. Gate + testthat green.
jaspPlot moves to src/core/ R-free: std::string ctor, plot object payloads travel through the jaspHost object store as opaque std::any handles, and rendering is delegated to jaspHost::renderPlot(jaspPlot&). The full R render pass (tryToWriteImageJaspResults incl. old-plot info, editing, interactive plotly, error handling) plus getPlotObject/toRObject/setUserPlotChanges move verbatim to adapters/rcpp/rcppPlot; jaspPlot_Interface (+setExport/getExport via jsonlite) moves to rcppInterfaces.h. jaspHost::plotStateSync re-applies user plot changes from the stored object after convertFromJSON (R: from _RStorageEnv). jaspResults state harvest uses rcppGetPlotObject; jaspObject gains clearError() + name() accessor for the adapter. Equivalence gate (fingerprint + 3 goldens + toRObject + state-store harvest) and testthat green.
jaspContainer moves to src/core/ R-free: std::string ctor, insert takes a core jaspObject* and core at() returns one, all tree admin, ordering, meta/dataEntry old-results merging, dependency pruning and JSON (un)packing verbatim. jaspObject gains virtual letRun()/complete() (table/plot/qmlSource mark theirs override) so the core can flip child status without including Rcpp headers. The Rcpp-facing pieces move to adapters/rcpp: the insert dispatch over the exposed *_Interface classes + R lists, jaspContainerFromRcppList, wrapJaspObject, at() and toRObject (rcppContainer), with jaspContainer_Interface into rcppInterfaces.h. jaspResults<Interface> adopts rcppInterfaces.h and the RDS branch calls rcppToRObject; letRun/complete/clairError markers adjusted where the base class gained virtuals. Also make runGate.sh wipe stale src/*.o/*.so before building: R's make does not track header dependencies and stale objects silently break vtable layout surfacing as bus errors at module-call time. Equivalence gate (clean build; fingerprint + 3 goldens + toRObject + state-store harvest) and testthat green.
…TableIngest
jaspTable and jaspList move to src/core/ R-free. The neutral cell storage
(std::vector<std::vector<Json::Value>> _data), all formatting/JSON/footnote/
schema/rowsJson/deriveColumnType logic, convertToJSON/convertFromJSON, and
dependency pruning stay verbatim in core. jaspList gains neutral insertIndex/
insertField/atIndex/atField/setRows(vector,map); jaspTable gains neutral
setColNames/setColTypes/etc. taking (vector, map) pairs, setDataColumns
(jaspTableData struct), addFootnote(string,string,vector,vector), and
addColumnInfo(string×5 + bool×2). The cell-storage primitives
(addOrSetColumnInData, pushbackToColumnInData, equalizeColumnsLengths,
setColumnCellsAt, setRowNamesWhereApplicable, getDesiredColumnIndex*,
deriveColumnType) are public so host adapters can build tables on top.
All Rcpp ingest dispatch (setData/setColumn/addColumns/addRows/addRow over
data.frame/list/matrix/vector with the exact Rcpp::is<...> chains), R-attribute
name extraction (extractRowNames, rownames/attr), the NULL-based
addColumnInfo/addFootnote, and toRObject (incl. mixed-column createMixedColumn
round-trip) move to adapters/rcpp/rcppTableIngest.{h,cpp}. jaspTable_Interface
and jaspList_Interface<T> (+ JASPLIST_MODULE_EXPORT macro, typedefs,
RCPP_EXPOSED) move to rcppInterfaces.h. rcppConversions gains
rcppListToRowsAndFields<T>. rcppToRObject dispatches table to
rcppTableToRObject.
Equivalence gate (clean build; fingerprint + 3 goldens + toRObject +
state-store harvest) and testthat green.
…nters jaspColumn moves to src/core/ R-free: the 13 column callbacks become plain function pointers (std::any payload for the three data setters), analysisId already lives on jaspHost. The R-facing XPtr registration moves to adapters/rcpp/rcppColumn (rcppSetColumnFuncs unwraps the engine's XPtrs and bridges the Rcpp::RObject data setters into the core std::any signatures; rcppCreateColumnsCPP keeps the deprecated R entry point). jaspColumn_Interface moves to rcppInterfaces.h wrapping payloads as std::any(Rcpp::RObject). Module names/signatures unchanged. Equivalence gate (clean build; fingerprint + 3 goldens + toRObject + state-store harvest) and testthat green.
…est, XPtr setters -> rcppResults
The last R-coupled class moves to src/core as R-free code: neutral
jaspPlotStateEntry/state-envName harvests, loadResultsIfLastWriteWorked()
split off the constructor, RDS branch and abort call behind jaspHost
callbacks (saveStateArchive now takes the jaspResults too).
New src/adapters/rcpp/rcppResults.{h,cpp} keeps the R half verbatim:
_RStorageEnv lifecycle, fillEnvironmentWithStateObjects, harvest ->
Rcpp::List rebuilders with the exact shapes common.R depends on, XPtr
unwrappers and jaspResults_Interface. jaspModuleRegistration.h moves to
adapters/rcpp too (with signature-preserving wrappers so the module
fingerprint stays byte-identical). Gate passes: fingerprint, all four
golden JSONs, toRObject + state-store baselines; testthat green.
…ull guard Three latent bugs found by the Phase-0 spike: 1. jaspPlot::_aspectRatio/_width/_height were uninitialized until the first render/convertFromJSON pass, so fresh plots serialized garbage into dataEntry/convertToJSON. Init to 0. 2. jaspState::dataToString printed "object stored: no" when the object existed (and vice versa); this was kept bug-for-bug during the moves, fixed now. 3. The R _RStorageEnv used to be created only in the jaspResults constructor, so jaspState/jaspPlot store access before any jaspResults existed dereferenced nullptr. The env is now created lazily on first use with the same inside/outside-JASP selection. stateStoreBaseline.R gains regression coverage for all three (pre-results round-trip, fresh-plot defaults, "object stored" print text via a subprocess probe); its fingerprint fixture is regenerated, the golden JSONs are byte-identical (none contained the affected outputs). Gate passes; testthat green.
…ase golden-table matrix
python/: scikit-build-core package compiling the real src/core with pybind11
(pyModule.cpp bindings, pyConversions implements the plan's cell contract:
None/nan -> "", inf -> ∞, NaNString marker for R's NaN cell, mixed-cell
dicts {value,type,format}, HTML escaping per table flag; ingest dispatch:
dict/DataFrame -> columns, Series -> named column, 2-D ndarray -> columns,
flat list/range -> one row, list-of-lists -> rows, Categorical -> strings).
python/tests/test_ingest.py: 20-case pytest matrix, all green.
tests/equivalence/tableGoldens.{R,py} + runTableGoldens.sh: 20 paired table
cases (cell types, NA/NaN/Inf, escaping, mixed columns via createMixedColumn,
footnotes, addColumnInfo, addRow/addRows/addColumns, transpose, expected size,
matrix/vector/DataFrame/Categorical ingest) built through the R module and the
pybind module; all 40 outputs (results JSON + toHtml) byte-identical.
Documented divergences encoded in the cases: R data.frame implicit rownames
must be passed as row_names in Python; R NaN needs the NaNString marker.
Phase-1 R gate (runGate.sh) still passes unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.