Evolving Mode Support: Auditing Capabilities

[Stuckya]

Intro

Embabel Agent already replans as actions add typed objects to the blackboard, and existing goals can become achievable during a running process. What's missing is an explicit goal lifecycle that lets a process evolve. These goals should complete without ending the process and become eligible again for a later occurrence. This matters most in long-running, event-driven processes.

I've coined 2 terms:

• Deterministic Evolving. Deterministic evolving is when a runtime fact drives a known goal to completion without ending the process, and a later fact can run it again. The fact-to-goal connection is typed and known ahead of time.
• Open Evolving. Open evolving aims to solve the case where not every fact-to-goal connection can be known and wired ahead of time. Instead of failing, open evolving attempts to solve the problem on-the-fly. Here, an ObjectiveAuthor proposes a revised policy.

My read on Evolving Mode is that declared agent metadata should remain immutable, and we shouldn't introduce a new PlannerType. We should reuse as much of the machinery that already exists within Embabel today.

Motivating Example

A concrete event-driven process might have a very long-running objective:

As a robot, collect samples in Zone A until 500 samples are stored.

The process runs over scoped capabilities / agents:

• sample collection
• robot navigation
• sample storage
• hazard response
• sensor calibration

The traditional Embabel pieces:

• declared actions handle collection, navigation, and storage
• declared goals are satisfied by normal action outputs such as SamplesStored
• conditions control availability and action selection

For example, sampleTrayFull is ongoing state. It stays true while the robot's sample tray is full and becomes false after the samples are stored. It should be modeled with @Condition and action preconditions, not as a runtime goal:

@Condition(name = "sampleTrayFull")
boolean sampleTrayFull(SampleTraySnapshot tray) {
    return tray.count() >= tray.capacity();
}

@Action(pre = "sampleTrayFull")
AtStorage navigateToStorage(CurrentLocation current, StorageLocation storage) {
    // navigate to the storage area
}

In this example, sampleTrayFull only gates whether storage/navigation actions are available. Evolving Mode is not involved merely because the tray is full.

Deterministic Evolving takes over when a fact should run a goal episode:

• SensorCalibrationRequested
• UrgentHazardDetected

Other facts can remain ordinary blackboard facts that completion policies, conditions, or actions read:

• SampleCollected
• SamplesStored

This is the sketched API:

var process = EvolvingInvocation.on(agentPlatform)
    .withScope(AgentScopeBuilder.fromInstances(
        new SampleCollection(),
        new Navigation(),
        new SampleStorage(),
        new Calibration(),
        new HazardResponse()))
    .withProcessOptions(
        ProcessOptions.DEFAULT.withPlannerType(PlannerType.HYBRID))
    .withEpisodes(EpisodePolicy
        .episode(GoalTarget.output(SensorCalibrationCompleted.class))
            .consumeOnCompletion(SensorCalibrationRequested.class)
        .episode(GoalTarget.output(HazardHandled.class))
            .consumeOnCompletion(UrgentHazardDetected.class)
            // Cooperative interruption of the running action,
            // backed by the existing AgentProcess.terminateAction().
            .terminateCurrentAction())
    // A declared condition-gated terminal goal in the scope retains
    // existing goal-completes-process behavior at samplesStored >= 500.
    .createAgentProcess(new CollectSamples("Zone A", 500));

agentPlatform.start(process);

process.ingress().publish(new SensorCalibrationRequested("cal-123"));

EvolvingInvocation and its fluent chain are proposed syntactical sugar. The scope, planner, conditions, goals, and BB behavior around them exist on main.

Stripping away the syntactical sugar, we could make it even simpler:

// The core: one new field on ProcessOptions, following the withers that already exist
var options = ProcessOptions.DEFAULT
    .withPlannerType(PlannerType.HYBRID)
    .withEpisodes(
        /* same EpisodePolicy as above */
    );

// Any existing invocation path now supports episodes. No new invocation type required:
var agent = AgentScopeBuilder.fromInstances(
        new SampleCollection(),
        new Navigation(),
        new SampleStorage(),
        new Calibration(),
        new HazardResponse())
    .createAgentScope()
    .createAgent(...);

var process = agentPlatform.createAgentProcessFrom(agent, options, new CollectSamples("Zone A", 500));

agentPlatform.start(process);

process.ingress().publish(new SensorCalibrationRequested("cal-123"));

Everything in this sketch except withEpisodes / EpisodePolicy, GoalTarget, and ingress() exists on main today.

Fact Lifecycle (runtime logic)

The framework recognizes satisfaction at a planning tick, the same point where a process completes today.

On completion of an episode's goal, the triggering fact and the satisfying output are hidden. Nothing is removed, the BB stays append-only. Hiding resets the goal to unsatisfied so the next occurrence plans work.

The lifecycle is per-occurrence. The goal is reusable. Completing an episode never completes the process.

Ordinary declared goals keep today's behavior. A condition-gated terminal goal still stops the process. withCompletionPolicy() can come later as ergonomics on top.

API should follow Embabel's existing patterns.

Episode selection uses existing conditions and plan values. No priority mechanism.

GoalTarget.output(T) matching several goals is one activation with candidate goals, and the first completion consumes it. GoalTarget.named() is the exact reference. A target no scoped goal can produce fails fast.

What's Missing in Infrastructure

The machinery exists. The contracts don't.

1. Nonterminal goal completion. Today whether a process survives a completed goal depends on plan-value ordering, and it flips between planners. ==> Solved by episodes (phase 1). EpisodePolicy marks selected goals as nonterminal.
2. Fact lifecycle. Could be hand-rolled today with hide(), a janitor action, and value tuning. The framework should own it. ==> Solved by episodes (phase 1). consumeOnCompletion hides the triggering fact and satisfying output so a later occurrence plans work.
3. Platform-owned wake. waitFor already parks a process WAITING for a promised fact. The resume is manual today. ==> Solved by ingress (phase 2). Publish enqueues the fact and the platform re-runs the process, reusing the existing Awaitable delivery path. Combined with episodes, this covers event-driven pure GOAP.

Baseline tests documenting current behavior under HYBRID and pure GOAP: https://github.com/Stuckya/embabel-agent/blob/goal-episode-baseline-tests/embabel-agent-api/src/test/kotlin/com/embabel/plan/GoalEpisodeBaselineTest.kt

Ordering of Work

My suggested ordering of the work.

1. Episodes from internal action-produced facts.
2. External fact ingress.
3. Observability.
4. Cooperative interruption.
5. Open Evolving, ObjectiveAuthor re-authoring.
6. Process-local scope expansion.
7. Native long-running pure GOAP. Recurring goal episodes.
8. An example application.

[igordayen]

@Stuckya - thanks for taking on this task. It's a significant effort! As a reader, I would find it very helpful if you could provide a shorter/simplified version of the story (preferably with minimal AI usage): key objectives and a suggested route. thanks again

[igordayen]

@Stuckya - hello, no activity recently there; could you please advise, thanks.

[Stuckya]

Hi Igor, I'll take a stab at this later today. Thanks!

[Stuckya]

@igordayen - I've re-written the issue entirely by hand. I've kept the longer spec locally, so we can feed it into an agent for implementation at a later date.

[igordayen]

@Stuckya - thanks very much for the next stub.

May I inquire:

In:

.onFact(SensorCalibrationRequested.class) ==> new object in Blackboard
.handleWithGoal(GoalTarget.output(SensorCalibrationCompleted.class)) ==> trigger anynimous goal ?

So, on "tick," monitor changes in BB, and trigger a goal?

[Stuckya]

Of course you can inquire.

I was thinking GoalTarget.output(T) would resolve to exactly one declared goal. Not anonymous.

onFact(SensorCalibrationRequested.class) would show up on the BB. Via an action output or published through the sketched .ingress().publish(). An observation matching a rule activates its runtime goal at a planning tick and normal planning takes it from there.

I think that not making it anonymous makes this safer and more declarative, but I could be mistaken. The target either resolves to a declared goal or fails fast, nothing gets invented at runtime. I'm a sucker for early returns and failing fast, but I'm happy to talk this through if you have an opinion!

[igordayen]

I was thinking GoalTarget.output(T) would resolve to exactly one declared goal. Not anonymous.

Good call.
Could match more than a single goal in general.
The agent can use heuristics/conditions to narrow the target goal.

RE: "Via an action output or published through the sketched .ingress().publish(). An observation matching a rule activates its runtime goal at a planning tick and normal planning takes it from there." ==> Perhaps, let's start with action output?

[igordayen]

I'm still trying to understand more:

handleWithGoal(GoalTarget.output(SensorCalibrationCompleted.class)) ==> existing goal.? What planner behavior gets modified then?
Apparently, I'm missing an important detail; sorry if my inquiry is trivial to you.
Thanks.

[Stuckya]

Good call.
Could match more than a single goal in general.
The agent can use heuristics/conditions to narrow the target goal.

RE: "Via an action output or published through the sketched .ingress().publish(). An observation matching a rule activates its runtime goal at a planning tick and normal planning takes it from there." ==> Perhaps, let's start with action output?

Couldn't agree more. This is currently prioritized as # 1 in the issue.

Re: heuristics
I'd save heuristics for open mode. An ObjectiveAuthor can use judgment to pick which goal, if multiple match.
For deterministic mode, we can tune with @Condition. We fail fast when no goal can produce the target.

I'm still trying to understand more:

handleWithGoal(GoalTarget.output(SensorCalibrationCompleted.class)) ==> existing goal.? What planner behavior gets modified then?
Apparently, I'm missing an important detail; sorry if my inquiry is trivial to you.
Thanks.

It isn't trivial to me either, hah! You are correct, it should resolve to an existing goal. In my interpretation, Evolving Mode does not change the planner algorithm.

At a planning tick, the planner sees all the existing metadata, but also can now plan toward SensorCalibrationCompleted.class candidate goals. Once the goal is achieved, the runtime goal is removed and the triggering SensorCalibrationRequested fact is hidden using the existing BB API. A later action can activate the rule again. The implementation should also ensure the previous completion cannot immediately satisfy that later activation.

The tradeoff is that one fact observation cannot fan-out to activate multiple runtime goals simultaneously:

UrgentHazardDetected
  -> HazardHandled
  -> IncidentReported

Additionally, completing the runtime goal shouldn't complete the process. In my sketch, completion belongs to the completion policy, withCompletionPolicy().

There is a question here for you. Should an evolving process only complete when its completion policy is satisfied? Or, should ordinary declared goals (under HYBRID / GOAP) complete the process?

My use-case benefits from Evolving Mode only completing when the completion policy is satisfied.

[Stuckya]

Also. I just noticed #1775 . This is very aligned with my thinking. Having multiple @AcheivesGoal is the same shape a rule target could encounter when a fact is produced.

#1775 sets the precedent, providing an emerging pattern to handle this.

Example:

Rule: onFact(SensorCalibrationRequested.class).handleWithGoal(GoalTarget.output(SensorCalibrationCompleted.class))

@Action(pre = "fieldConditionsOk")
@AchievesGoal(description = "quick field calibration")
SensorCalibrationCompleted quickCal(...) { }

@Action(pre = "atMaintenanceBay")
@AchievesGoal(description = "full bay calibration")
SensorCalibrationCompleted fullCal(...) { }

Both goals match. I'd treat that as one activation with 2 candidate goals. Conditions steer which candidates are plannable at the tick. The first completion consumes the activation.

[igordayen]

@Stuckya - my understanding is still evolving in Evolving mode:)

withScope(AgentScopeBuilder.fromInstances(
        new SampleCollection(), new Navigation(), new SampleStorage(), new HazardResponse()))
    .withProcessOptions(ProcessOptions.DEFAULT.withPlannerType(PlannerType.HYBRID))   // standing collection + runtime goals
    .withEvolution(EvolutionPolicy
        .onFact(SensorCalibrationRequested.class)

in the fragment above:

• new SampleCollection(), new Navigation(), new SampleStorage(), new HazardResponse())) - starting objects ib blackboard
• SensorCalibrationRequested.class - new object created either as a result of an action, or due to event capturing

How do the semantics differ from the normal case when an action creates a new object in BB?

Is the key of the proposal - once the goal gets satisfied - remove the object from BB (or ide it)?

Re: "Additionally, completing the runtime goal shouldn't complete the process. In my sketch, completion belongs to the completion policy, withCompletionPolicy().

There is a question here for you. Should an evolving process only complete when its completion policy is satisfied? Or, should ordinary declared goals (under HYBRID / GOAP) complete the process?" ==> not obvious to me, perhaps can be controlled by config param?

[igordayen]

Both goals match. I'd treat that as one activation with 2 candidate goals. Conditions steer which candidates are plannable at the tick. The first completion consumes the activation.

Yes, correct. see embabel-cookbook for examples, including heuristics.

[Stuckya]

new SampleCollection(), new Navigation(), new SampleStorage(), new HazardResponse())) - starting objects ib blackboard

Small correction on the first bullet: fromInstances(...) takes agent/component instances. @Action, @Condition, and @AchievesGoal metadata is used to compose the scope. They are not blackboard objects. The initial blackboard input here is new CollectSamples("Zone A", 500).

How do the semantics differ from the normal case when an action creates a new object in BB?

Adding the object to the BB is identical. The difference is what happens at the planning tick.

1. An observation matching a rule changes what the process can pursue. The target goal candidates aren’t active until the rule observes the fact.
2. Blackboard objects normally remain visible. Without lifecycle management, the same request could be matched repeatedly. A later SensorCalibrationRequested could also cause the planner to see an old SensorCalibrationCompleted output and treat the goal as already satisfied. The rule supplies activate-once, complete-once, consume, and rearm behavior.
3. Under HYBRID/GOAP today, satisfying a real goal completes the process. Under an evolving rule, completing the runtime goal consumes that activation and the run continues.

Is the key of the proposal - once the goal gets satisfied - remove the object from BB (or ide it)?

Yes, that's one key of the proposal. (If I'm not mistaken BB has no remove and is append-only)
The other two keys are goal activation and the process outliving goal completion.

not obvious to me, perhaps can be controlled by config param?

Agreed. It should be controllable. I’d probably just hang it off the invocation rather than platform configuration, since two processes on one platform may want different behavior. Perhaps existing goal-completes-process behavior remains the default unless withCompletionPolicy(...) is supplied? We may form a stronger opinion while implementing it.