|
1 | | -# Growth research compatibility application |
| 1 | +# Growth research application |
| 2 | + |
| 3 | +## Local company research pilot |
| 4 | + |
| 5 | +The local pilot compares one bounded Dawn agent with the existing lifecycle enrichment |
| 6 | +generator on identical captured company evidence. It has no Growth database connection, |
| 7 | +does not resolve people or employment, and cannot send email. The managed deployment |
| 8 | +still exposes only the synthetic compatibility graph documented below. Pilot routes, |
| 9 | +operator adapters, and their generated graph are excluded from its staged artifact. |
| 10 | + |
| 11 | +Use Node 24 and the existing workspace dependencies. Build before running the agent: |
| 12 | + |
| 13 | +```sh |
| 14 | +npx nx build growth-research |
| 15 | +npx tsx apps/growth-research/scripts/research-pilot.mts synthetic --output /absolute/private/pilot |
| 16 | +npx tsx apps/growth-research/scripts/research-pilot.mts acquire --output /absolute/private/pilot --domains threadplane.ai,dawnai.org,neon.tech,vercel.com,resend.com,langchain.com |
| 17 | +``` |
| 18 | + |
| 19 | +These commands return UUIDs for immutable JSON files in the selected output directory. |
| 20 | +Acquisition records include complete, partial, empty and failed outcomes. The existing |
| 21 | +fetcher can skip unusable pages, so missing paths have an unknown reason; redirects may |
| 22 | +make the original path indeterminate. Review the captured corpus before model calls: |
| 23 | +remove personal biography/contact snippets, retain empty cases and failures, and fill |
| 24 | +expected claims/unknowns from the actual captured evidence. Save the reviewed corpus |
| 25 | +under a new name/version. Acquisition is preparation, not a human quality label. |
| 26 | + |
| 27 | +Set `GROWTH_RESEARCH_PILOT_MODE=local-company-only` and configure `OPENAI_API_KEY` |
| 28 | +for the agent or `ANTHROPIC_API_KEY` for the baseline through the operator environment. |
| 29 | +Never include keys in arguments, fixtures, reports or commits. The local in-process |
| 30 | +case context is also required: an environment flag alone cannot authorize pilot tools. |
| 31 | + |
| 32 | +```sh |
| 33 | +npx tsx apps/growth-research/scripts/research-pilot.mts run --output /absolute/private/pilot --corpus /absolute/private/pilot/CORPUS_UUID.json --approach agent |
| 34 | +npx tsx apps/growth-research/scripts/research-pilot.mts run --output /absolute/private/pilot --corpus /absolute/private/pilot/CORPUS_UUID.json --approach baseline |
| 35 | +npx tsx apps/growth-research/scripts/research-pilot.mts inspect --output /absolute/private/pilot --run RUN_UUID |
| 36 | +``` |
| 37 | + |
| 38 | +Each case/approach/repetition has a separate run ID, deadline, budget and terminal |
| 39 | +record. Runs execute sequentially. The agent permits six provider requests, six evidence |
| 40 | +reads, 1,024 output tokens per request, no provider retries, a 20-second request timeout, |
| 41 | +and a 90-second run deadline. It can use the evidence skill and plan, read only captured |
| 42 | +case evidence, and submit a candidate. It cannot delegate, use memory, fetch URLs or |
| 43 | +read arbitrary files. Candidate acceptance is structural validation, not a truth label. |
| 44 | +Explicit inspection shows company sources and candidate findings; ordinary progress |
| 45 | +prints only opaque IDs and outcome codes. Reports use restrictive atomic writes and |
| 46 | +refuse overwrites. Preserve the final index and all failed attempts when comparing runs. |
| 47 | + |
| 48 | +The baseline uses its existing provider/model and 1,200-token/30-second request bounds. |
| 49 | +It receives company mode, synthetic adapter form context and zero progress score. |
| 50 | +Its raw citations are captured before production normalization. It does not return |
| 51 | +quotes: `not_provided` is distinct from failing or passing exact-quote validation. |
| 52 | +Provider failure records retain known request/usage/citation diagnostics. Missing usage |
| 53 | +and cost are unavailable, never zero. This comparison measures whole approaches with |
| 54 | +different providers/models; it does not isolate Dawn's causal contribution. |
| 55 | + |
| 56 | +Raw automatic tracing is disabled for local pilot runs. The record reports |
| 57 | +`tracing: unavailable`; this slice does not claim sanitized LangSmith tracing is live. |
| 58 | +No research findings are automatically published to Growth or typed memory. |
| 59 | + |
| 60 | +### Human comparison |
| 61 | + |
| 62 | +Each invocation emits a blinded review packet. To combine baseline and agent results |
| 63 | +for the same corpus, pass their index UUIDs; mixed corpus hashes/classes are rejected: |
| 64 | + |
| 65 | +```sh |
| 66 | +npx tsx apps/growth-research/scripts/research-pilot.mts review --output /absolute/private/pilot --indices BASELINE_INDEX_UUID,AGENT_INDEX_UUID |
| 67 | +npx tsx apps/growth-research/scripts/research-pilot.mts score --output /absolute/private/pilot --packet PACKET_UUID --labels /absolute/private/pilot/human-labels.json |
| 68 | +``` |
| 69 | + |
| 70 | +The packet omits model and approach labels. Reviewers inspect each claim and profile |
| 71 | +against the captured sources, including failed cases. `human-labels.json` is an array: |
| 72 | + |
| 73 | +```json |
| 74 | +[{"reviewId":"RUN_UUID","supportedClaims":0,"reviewedClaims":0,"supportedFields":0,"applicableFields":0,"correctAbstentions":3,"applicableAbstentions":3,"contradictionsMissed":0}] |
| 75 | +``` |
| 76 | + |
| 77 | +Use actual UUIDs and counts for each case. Reviewed claim count must match the packet; |
| 78 | +applicable fields and abstentions come from its expected unknowns. Imported labels and |
| 79 | +per-approach scores are persisted as a new review artifact. Aggregate quality scores |
| 80 | +remain unavailable while reviews are incomplete, preventing success-only denominators. |
| 81 | +Human semantic review is not replaced by model grading or string matching. |
| 82 | + |
| 83 | +### Dogfooding findings ledger |
| 84 | + |
| 85 | +| Finding | Evidence / owning layer | Status and next verification | |
| 86 | +| --- | --- | --- | |
| 87 | +| Nullable tool fields become required strings | Dawn 0.8.24 compiler JSON schema conversion; observed generated submit schema and failed unknown-field submissions | Upstream core and LangChain conversion regression/fix in progress. Pilot uses the supported authored Zod schema export; a package upgrade must rerun the original extraction probe before declaring the upstream defect released. | |
| 88 | +| Bound model calls bypass subclass generation hooks | Real bound-model regression in this application | Guards, request counts and JSON usage capture live at the actual provider fetch boundary; generated graph tests verify it. | |
| 89 | +| Page capture yields empty, partial, or mostly navigation evidence | Company-only acquisition against the six documented domains | Outcomes retained. Evaluate extraction improvements separately; do not hide failures by swapping cases. | |
| 90 | +| Baseline provider rejects billing state | Live baseline synthetic calls returned a classified billing rejection | External provider funding/configuration required; no quality comparison can be claimed from failed calls. | |
| 91 | +| Managed interruption precedes later child checkpoint | Recorded local/cloud Agent Server 0.13.4-node24 probe | Still a live-person integration gate; local cancellation tests are not proof of managed cancellation. | |
| 92 | +| Disabled memory and shared harness persistence behavior | Earlier synthetic compatibility probe on Dawn 0.8.24 | Reproduction-needed against current Dawn before assigning a fix. Pilot has no memory and graph tests use isolated state. | |
| 93 | + |
| 94 | +Keep source snapshots, generated reports and review labels outside git. The full growth |
| 95 | +funnel/contact journey and real install/runtime-triggered enrichment are subsequent |
| 96 | +slices, after supported company context and the managed data lifecycle are verified. |
| 97 | + |
| 98 | +## Synthetic compatibility deployment |
2 | 99 |
|
3 | 100 | Private synthetic Dawn application, separate from lifecycle and the Python cockpit. |
4 | 101 | Published Dawn packages are pinned to `0.8.24`; the app and deployment require Node 24. |
|
0 commit comments