Skip to content

CamiViolet/qqfour

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 

Repository files navigation

qqFour

qqFour is a human-curated, AI-augmented knowledge base system designed to support information handling, thinking, exploration, and daily engineering work without relying on automated indexing or large-scale retrieval pipelines.

It is intended for individual developers or small software teams in a context of a large company with multiple teams and projects.

Example use cases can be:

  • Handing over of knowledge about a project, a topic, or a task.
  • Supporting explorative tasks around a new topic.
  • Learning, by allowing an incremental enrichment of the knowledge base and individual know-how over time.
  • Supporting day-by-day work, like reviews, design, brainstorming, etc.

The Concept

qqFour is at the same time a tool, a workflow, and a paradigm. It is important to understand that the user is responsible for ensuring the quality of the knowledge base.

The pillars of the qqFour's paradigm are the knowledge base, the iterative workflow, and the tools.

The knowledge base is a collection of documents that is composed of three key parts: the imports directory, the log.md, the activities.

The imports directory contains the files imported from multiple sources. Each file is tagged so that it can be easily retrieved, but apart from this the directory is unstructured. It's the main source of information.

The log.md file is a chronological list of individual notes. It is essential to guarantee the richness of the knowledge base.

The activities directory contains a number of independent activities, identified by the name of the directory. Each activity addresses a specific topic, project, or task. For example, an activity can be a design review, a brainstorming session, an exploration of a new topic, etc.

When working on an activity, the user shall execute iteratively the following operations:

  • Ask questions to the LLM by using the context provided by the activity.
  • Review and improve the existing .md files or create new ones.
  • Include more information from the imports in the activity.
  • Add new documents to the imports directory.

customized mental model, the information density, and the scalable structure of the knowledge base.

What qqFour is not

qqFour is not a RAG tool. Specifically, it does not apply any offline indexing of the knowledge base nor any other kind of pipeline.

Why qqFour

The customized mental model. In the traditional information flow, the author of the documents chooses how to organise the information, how ideas connect, and the order in which everything should be read. The documentation is often written with implicit assumptions, contextual knowledge, and unstated relationships. But readers may approach the content following different mental paths, and may need the information grouped or combined differently. As a result, the superficial structure of the documents becomes an obstacle. This is why readers are rarely fully satisfied with documentation, no matter how well it’s written. With qqFour, the readers compose the information in the way that matches their own reasoning.

The consistency and focus. By relying on a manually curated knowledge base, qqFour ensures that the information is consistent and focused on the relevant topics. For example, an automatic crawling may retrieve contradictory information related to different versions of the same software, or it may retrieve a decision that later on has been changed.

The information density. The log and the manually created document must contain pros and cons, hypotheses, assumptions, considerations, doubts, and counter-arguments. This makes the information semantically rich, integrated and connected, regardless of the superficial structure of the documents. This allows the LLM to provide accurate and relevant answers.

The scalable structure. The scalable structure of the imports and activities directories allows to manage dynamic and evolving contexts. It also allows a distributed management among the team members.

The Knowledge Base

The knowledge base is a collection of documents. Usually, a development team uses one single knowledge base. It is a living resource that requires the user to correct mistakes, add new information, and remove obsolete content. It is composed of three key parts: the imports directory, the log.md, the activities.

The imports directory contains the imported files, in markdown format, from multiple sources. Example sources can be email chats, meeting transcripts, Jira tickets, Confluence pages, PDF documents, etc. The user must ensure that only relevant and high-quality content is included. Automatic crawling cannot be used. This is essential to keep the knowledge base consistent and focused.

The knowledge base is composed only by plain text file or by Markdown files. Diagrams are embedded directly in the Markdown documents and use human‑readable languages such as Mermaid or PlantUML. This allows a smoother workflow between the author and the generative AI. For example, the LLM can easily modify the diagrams.

The qqFour's paradigm assumes that the user explicitly maintains the knowledge base by keeping the clean and focused. Automatic crawling is not used.

This brings several advantages:

  • The knowledge base must be kept small, so that it can be searched in real-time. This means that any information is available as soon as it is added to the knowledge base. Furthermore, this minimizes the usage of tokens.
  • The knowledge base is much more dense of information because it is manually curated. This means that the LLM can provide more accurate and relevant answers.
  • For the same reason it has lower risk of retrieving obsolete or inconsistent information.

Inside the knowledge base, you find the following directory structure:

kbx/
├── log.md
├── activities/
└── imports/

The log.md is a manually maintained file that contains continuous updates on all subjects related to the knowledge base. The incremental content is added from the top. The log.md is continuously updated by the user by adding new information, insights, ideas, etc. The content is added from the top. Each new note must be self-contained and well contextualized. They are usually short, ranging from a single line to a few dozen. Notes often present pros and cons, hypotheses, assumptions, considerations, doubts, and counter-arguments, which makes them semantically very “dense”. The overall content of the file is fragmented and unstructured, but nevertheless, very dense.

Each Activity is typically used to track day-to-day work, ongoing tasks, ideas, or creative exploration. The activities directory has the following structure:

<name_of_the_activity>/
├── description.md                  # Description of the activity
├── <name>.md                       # Manually crafted documents. This is the 'output' of the activity
├── raw/
│   └── <name>.gitignore.1bf.txt    # Generated supporting files
├── 1bf_configs/
│   └── 1bf_config_<name>.json      # Configuration for the generation of the supporting files
└── scripts/
    └── <script_name>.py            # Activity specific scripts

An 'activity' contains the following type of files:

  • description.md : It describes the purpose of the activity. It will be used by qqFour to help searching for the activities. This file is mandatory and it's created and maintained by the user.

  • <name>.md : This is the 'output' of the activity. It's crafted by the user. There can be many of these files.

  • <name>.gitignore.1bf.txt: A 1bf ("One Big Files") is an internal file generated by qqFour by using data from the imports directory. It contains fragmented, unstructured information, and it is not intended to be read or edited by the user.

  • 1bf_config_<name>.json : each one of these files configures the generation of a 1bf file. For example, it specifies which files must be collected from the imports directory. This files are created by the user.

  • <script_name>.py : Optional activity-specific scripts that elaborates the imported data. For example, it can be used to compose the material for the reviews.

The Workflow

The user continuously updates log.md with new information, insights, ideas, etc. The content is fragmented and unstructured. The essential point is that every note must be well contestualized by using proper wording.

The user shall handpick and manually control the content of the knowledge base. Only high-quality, relevant content is included. No automatic crawling.

Note that LLM works best with information-dense documents. For this reason manually added information is always enriched with rationales, assumptions, considerations. Add also consideration about pros and cons of every decision.

As a general rule, the .md files should never explode in number and size, because the user should always keep the informative content of the activity under control. The rule of thumb is that qqFour knows a more than the user, but not too much more.

The tool

qqFour is a command line shell.

  • It allows to import documents from different sources, like Jira, Confluence, PDF documents, etc.
  • It allows to search for existing 'activities'.
  • Specific functionalities prepares the context for LLM's chat by using the 'imports' or the 'activity' directory. The script make use of fragmenting techniques, deduplication, and it preserves the metadata. The search is dynamic as it is based on the content of the knowledge base at the moment of the search. No offline pipelines are used.
  • Generates the '1bf' files by using the information from the 'imports' directory.

The Keywords

The links between concepts are established by using keywords. This is different from the typical approach of using backlinks.

The keywords are metadata manually added to the files.

The relationship that can be defined may have different semantics.

For example, the Related relationship is weaker than the Keywords one. The Related relationship is used to link documents that are loosely connected or alternative topics. The Keywords relationship is used to link documents that are closely related and share the same main subjects.

The Context relationship defines the specific domain or environment to which the document applies, helping to narrow search relevance.

The Alternatives relationship lists concepts that are alternative to the one described in the document.

About

qqFour is a human-curated, AI-augmented knowledge base system designed to support information handling, thinking, exploration, and daily engineering work. It is intended for individual developers or small software teams in a context of a large company with multiple teams and projects.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors