move rgkl crate to rgkl module

[ArshpreetS]

Fixes #594

Summary

Converts rgkl crate to a module and moved to cluster_agent

Submitter checklist

• Add the correct emoji to the PR title
• Link the issue number, if any, to Fixes #
• Add summary and explain changes in the PR description
• Rebase branch to HEAD
• Squash changes into one signed, single commit [^1]

[ArshpreetS]

Status update: I have moved the rgkl crate into a module inside cluster_agent but I am facing some issues when I am running the crate test. I am facing trouble because different instances are giving different failed tests ( I changed nothing )

If you have some idea then let me know

[amorey]

What OS are you developing on? The tests are passing in the PR checks which use linux runners so my guess is that the errors you're seeing have to do with the RecommendedWatcher used by the notify crate for your OS: https://github.com/kubetail-org/kubetail/blob/main/crates/rgkl/src/stream_forward.rs#L238.

If notify uses a poll watcher then it can cause intermittent failures in the tests because the tests assume real-time notifications when a file is updated. This is ok for production because the cluster agent only targets linux but it's frustrating for development so we should either exclude the test from problematic OS's or update code so the tests pass on all OS's.

[amorey]

Did you consider mimicking the LogMetadata service and putting the code used by the LogRecords service into a log_records directory?

[ArshpreetS]

What OS are you developing on?

I am using macbook

[ArshpreetS]

Did you consider mimicking the LogMetadata service and putting the code used by the LogRecords service into a log_records directory?

oh I wasn't aware of this. Let me check this out

[ArshpreetS]

Did you consider mimicking the LogMetadata service and putting the code used by the LogRecords service into a log_records directory?

Why will that help?

[amorey]

Why will that help?

The Cluster Agent is a gRPC server with two services - LogMetadata and LogRecords. Currently, the LogMetadata service implementation code is organized in a sub-directory of the cluster_agent crate but the LogRecords service implemenation code is in the separate rgkl crate so moving it into a sub-directory would give the services a consistent architecture.