what are slots?

[siddhsql]

hello, could you tell me what are slots?

https://github.com/kherud/java-llama.cpp/blob/master/src/main/cpp/server.hpp#L106

struct server_slot
{
    int id;
    int id_task = -1;
    int id_multi = -1;

    struct slot_params params;

    slot_state state = SLOT_STATE_IDLE;
    slot_command command = SLOT_COMMAND_NONE;

    // used to determine the slot that has been used the longest
    int64_t t_last_used = -1;

    // generation props
    int32_t n_ctx = 0; // context size per slot
    int32_t n_past = 0;
    int32_t n_decoded = 0;
    int32_t n_remaining = -1;
    int32_t i_batch = -1;
    int32_t n_predict = -1; // T

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125

[bernardladenthin]

For reference: in the downstream fork at https://github.com/bernardladenthin/java-llama.cpp
the hand-ported src/main/cpp/server.hpp has been removed; the project now
compiles the upstream server-context.cpp / server-queue.cpp /
server-task.cpp / server-models.cpp directly. server_slot is therefore
an upstream-owned concept; callers see it indirectly via slot-related task
params (InferenceParameters.setIdSlot, etc.).