harden: use yaml.safe_load in generate_index_of_sub_section.py... - #398
Open
anupamme wants to merge 175 commits into
Open
harden: use yaml.safe_load in generate_index_of_sub_section.py...#398anupamme wants to merge 175 commits into
anupamme wants to merge 175 commits into
Conversation
Update/sof 7587 coordintaes copying
update/SOF 7691 1
chore: remove external uploads
chore/SOF 7666 - Migration to New Platform, New Infrastructure Specs
update: colors, icons, cleanups
…sion M5 is done and the agent is deployed and answering, so only the launch gate remains. Records what was verified against the deployed service rather than locally, including the rollback drill, and three findings worth not rediscovering: a commit SHA cannot be passed to git clone --branch, so pinning the index silently needed a different checkout; builds submit --tag cannot pass the build argument that pinning requires; and a brand-new project fails its first deploy on permission propagation rather than on any policy. The consequence for sequencing is now stated plainly. The service is live and the widget appears only once it answers a health check, so merging the widget to main is the launch, and it waits on the hardening gate rather than on anything technical. Marks D3 done - Vertex confirmed end to end from Cloud Run - and D8's refusal bar as reached rather than pending. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things the first real use made obvious. The Mat3ra mark now sits beside the "Ask AI" label in the launcher and the panel header. It is inlined and drawn in currentColor, so it works on the purple launcher and the light header without shipping two assets or depending on an image path that exists only on the documentation site. Source URLs are now links. The model lists them as bare URLs and the renderer only understood the bracketed Markdown form, so every citation arrived as dead text - a source you cannot click is not much of a citation. Emphasised product terms - Materials Bank, Materials Designer - now link to the page that defines them, which is what a reader wants to open. The mapping comes from the service, which derives it from its index, so a link can only ever name a page that exists; the model is never asked to produce one. Terms stay styled as emphasis with a dotted underline rather than turning the answer into a field of links. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sources and glossary terms opened in a new tab. A documentation link is the continuation of the answer rather than a detour, and spawning tabs behind the reader is a habit the documentation itself does not have. The consequence is worth stating: the conversation is held in memory only, so following a link ends it, and the back button returns to the documentation page rather than to the exchange. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sources and glossary terms rendered in the surrounding text colour, so nothing signalled they could be clicked. They now use the documentation's own link colour where the theme defines one, and a conventional blue wherever else the widget is embedded, since the service's try-it page and the platform shell have no such variable. This drops the earlier treatment of glossary terms as emphasis with a dotted underline. Reading as prose is worth less than being visibly clickable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Answers cite documentation pages and those links now open in place, so every citation was ending the exchange that produced it: the reader followed a source, the page reloaded, and the panel came back empty. The conversation now outlives the page. It is stored in the browser, restored on load, and the panel reopens if it was open, so following a link reads as continuing rather than starting over. Storage is bounded the same way the service bounds a request - the last twenty messages, capped by total size - and expires after a week, which is what makes "relive through multiple visits" tolerable rather than indefinite. A conversation that survives navigation also has to be endable, so the header gains "New chat" and the footer says plainly that the exchange is kept in the browser until then. Nothing new is sent anywhere: this is text the page already displayed, on the reader's own machine. Restoration waits for the glossary, so a restored answer gets the same links a fresh one would. Storage being full or disabled is not an error - the widget simply behaves as it did before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Following a source on a deploy preview jumped to production docs.mat3ra.com, because the corpus stores canonical URLs and the widget rendered them verbatim. That left the preview entirely, and took the conversation with it - stored sessions are per-origin, so the panel came back empty on the other side. A documentation URL is now rewritten to the origin currently serving the documentation, which is a no-op in production and makes previews and local servers behave like it. Rewriting only happens where the widget mounts itself onto a documentation build; the platform shell serves no documentation, so its citations keep pointing at the real site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The widget ships to every documentation page, and until now nothing guarded it. Each fix in it so far was verified by hand once and then left unprotected, which is how the same bug comes back. Seventeen Playwright tests run in under three seconds and need neither cloud access nor a model call. The widget under test is the file the site ships; only the agent service is faked, at the network boundary, because a test that called a language model would be neither deterministic nor free and the service has its own suite. The fixture server answers every path with a page that mounts the widget, which is what makes the navigation tests possible: answers cite canonical production URLs that the widget rewrites onto the origin being read, so following a citation has to land somewhere the widget mounts again. Both mounting styles are covered, since they differ - the documentation site lets the script mount itself only after a health check, while the platform will mount it explicitly. Coverage is deliberately weighted towards the properties that are easy to break silently: markup in an answer is displayed rather than executed, a non-https link never becomes clickable, unknown emphasis stays plain rather than linking somewhere plausible, and the conversation survives a followed citation without resurrecting a week-old one. Both regressions these tests were written for were reintroduced on purpose and confirmed to fail the run, so the suite is known to have teeth rather than assumed to. The README says to do the same when adding to it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
M4 is finished and now has browser tests, so the plan records what the first real use taught rather than only that it shipped: citations had to become clickable and same-tab, emphasised terms link through a glossary the service derives from its index, citations must stay on the build being read, and the conversation has to outlive the page once links open in place. Adds the local Google Cloud SDK to the outstanding items. It is still the June 2023 release, and Google rejects that client's token refresh, so a fresh login lasts about an hour before every call fails. Re-authenticating treats it hourly; upgrading fixes it. Nothing deployed depends on it, since Cloud Run authenticates as its own service account, but no deploy can be driven from a laptop in this state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three stacked causes, not the one the plan guessed: the machine driving builds is a different computer from the laptop whose logins kept not helping; its 2023-era gcloud minted tokens Google began rejecting outright on 2026-08-01; and the account session now requires interactive reauthentication, which no non-interactive shell can satisfy. Deploys run again - a current SDK from a local directory, fed tokens minted from application-default credentials, which still refresh. CI with federated identity bypasses the whole class of problem, which is one more reason to merge the pipeline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Feature/sof 7992 agent rag
…unified structure
feature/SOF 7969
Reproduces the electronic-structure result of Liu et al., Nat. Commun. 5, 4966 (2014): the indirect gap of a MoS2 bilayer is set by the interlayer distance, while the K-valley direct gap barely moves. Covers the prerequisite structure notebook, the structures computed, where the settings differ from the manuscript's and why, and the results. Registered in both mkdocs.yml and mkdocs-guide.yml, as the sibling simulation tutorial is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The out-of-plane cell is now 20 A as in the manuscript, so it is no longer a divergence. The cutoff row is: 140 Ry is a norm-conserving wavefunction cutoff and is not comparable with an ultrasoft one, which converges lower and needs a higher charge-density cutoff instead. Drop the walkthrough of the Mo-Mo to sulfur-plane conversion; the notebook prints the resulting separation and cell height for checking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The band-structure page did not read as the second half of the MoS2 manuscript reproduction: - nav titled it "Twisted Bilayer MoS2 Band Structure" instead of the structure page's title plus "(Band Structure)", the form already used for the graphene pair - in mkdocs-guide.yml it sat seven rows above its own structure page, directly under the graphene pair - overview.md still advertised its Properties entry as "Coming Soon" Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follows the _SIMULATION rename in api-examples. Two references in the page - the GitHub source path and the jupyterlite_embed notebook_name. The page slug is unchanged, so no published URL moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cost-lever paragraph still named INTERFACE_VACUUM, which no longer exists - the structure notebook takes TOTAL_CELL_HEIGHT and derives the builder's vacuum from it. It also framed the cell as more generous than the manuscript's, which stopped being true when it moved to 20 A. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sections 3 and 4 documented create_twisted_interface_with_commnesurate_ lattices.ipynb, while section 5 embeds specific_examples/interface_bilayer_twisted_commensurate_lattices_ molybdenum_disulfide.ipynb. None of the variables the page told the reader to set - TARGET_TWIST_ANGLE, INTERFACE_DISTANCE, INTERFACE_VACUUM, MAX_SUPERCELL_MATRIX_INT, USE_CONVENTIONAL_CELL, STACKING_DIRECTION - exist in the embedded notebook, so the prerequisite step of the band structure tutorial could not be followed as written. Rewritten against the embedded notebook: the INTERFACE_PARAMETERS list replaces the one-angle-at-a-time walkthrough, TOTAL_CELL_HEIGHT replaces INTERFACE_VACUUM, and the six angle snippets become uncommented entries. Two things the old text got wrong about the physics, now called out: d_mo_mo is the Mo-Mo separation Table S1 tabulates, not a gap - passing 6.5 A through as a gap separates the layers ~3 A too far; and the interlayer distance belongs to the stacking registry, 6.1-6.2 A for AA1/AB1 and AA2/AB2 against 6.8 A for AA3/AB3. Atom counts, Mo-Mo separations, cell heights and the two registry classifications in the new text were built and measured, not inferred. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SOF-8014: tutorial for twisted MoS2 bilayer band structure
Automated security fix generated by OrbisAI Security
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Harden input handling in
scripts/generate_index_of_sub_section.py(flagged by semgrep).Vulnerability
gitlab.bandit.B506scripts/generate_index_of_sub_section.py:61Description: The application was found using an unsafe version of
yamlload which is vulnerable todeserialization attacks. Deserialization attacks exploit the process of reading serialized
data and turning it back
into an object. By constructing malicious objects and serializing them, an adversary may
attempt to:
deserialization process.
data but are read in during deserialization.
To remediate this issue, use
safe_load()or callyaml.load()with theLoaderargumentset to
yaml.SafeLoader.Example loading YAML using
safe_load:For more details on deserialization attacks in general, see OWASP's guide:
Threat Model Context
This is a web service - vulnerabilities in request handlers are directly exploitable by remote attackers.
Changes
scripts/generate_index_of_sub_section.pyBehavior Preservation
The change is scoped to 1 file on the vulnerable path; it only tightens handling of untrusted input and leaves valid inputs unaffected.
This patch removes an exploit primitive — a code pattern that, while not independently exploitable today, could be chained with other weaknesses by automated exploit-development tooling. Proactive removal of such primitives raises the bar against increasingly capable automated attack tools.
Automated security fix by OrbisAI Security