recipes-support: Add new recipe for tftp-server#1445
recipes-support: Add new recipe for tftp-server#1445Pradeep-pvk wants to merge 4 commits intoqualcomm-linux:masterfrom
Conversation
Pradeep-pvk
requested review from
lumag,
ndechesne,
ricardosalveti,
sbanerjee-quic and
vkraleti
as code owners
lumag
left a comment
lumag
left a comment
There was a problem hiding this comment.
It doesn't specify, why the binary is better than the open-source one
In general, I would not like to have it here.
The currently available open‑source binary “tqftpserv” does not yet support the full set of functionalities required to serve RFS requests. Development is still ongoing, and the missing features may not be completed until late this year. This delay could impact our targets and product timelines for QLI 2.0. Therefore, we propose using the downstream binary “tftp_server” from Artifactory in the interim. |
|
Please point out the ticket on tqftpserv describing the issue |
Here is the ticket for the ongoing tqftpserv development work: |
It's not a ticket. It s a PR for adding syslog and a conf file, which has nothing to do with the issue that you've mentioned above:
In addition to this, there are few other items and features planned in the pipeline, such as multithreading support, TFTP windowing, and others. When you've dropped an email several days ago, you have been asked to provide description of the features you consider to be missing (your email had very brief sentences for them). As you've moved the discussion to GitHub, please fill the issues against the tqftpserv, describing the issue, impact and providing other necessary information. Until we can assess what is missing, tqftpserv is the preferred solution, the closed source implementation is rejected. |
i didn't get what do you mean by ticket then
AFAIR, i didn't dropped any email. i will check with my team if they do. BTW, did you tested "tqftpserv" on qualcomm targets and is it serving the RFS requests from ADSP and MPSS clients in the expected way ? |
Issue on the GitHub, describing the problem.
Dropped = sent Up to now, tqftpserv has been working correctly enough in our environment. It has been in use by robotics platforms, by several phones, etc. |
|
Also, why are we providing it as binary while we can probably make the source code available instead? |
At present, the source code remains proprietary and is not shipped. |
|
I was still wondering how this works—since the data is stored in /tmp (which is non‑persistent), how does it persist across reboots? Does that mean there is no real write use case executed on this platform? |
Yes. |
Please respond to Bjorn's questions. |
So, it is now clear that the existing tqftpserv cannot preserve data. We need an alternative until this is fixed. otherwise, targets running QLI 2.0 will be impacted. Please consider this overlay in the meantime and help with the review. |
Sure, but please do take a moment to review these changes. |
|
Regardless of the discussion about the open source vs closed alternative, can you please make sure the submitted PR at least passes the CI we are running? we cannot do anything with PR which fails the CI. |
What is the usecase for preserving the data? Which of the services uses it in the real world? |
Here are few:
|
This sounds like an optimization.
Modem-enabled configurations are not on the top list. In general it sounds like an important feature. Please discuss persistence at the corresponding tqftpserv ticket (and provide a suggested implementation). We will upgrade tqftpserv here as soon as it is released. |
Add new prebuilt recipe for tftp-server to fetch Qualcomm properitary userspace application from artifactory. This application serves the Remote Files System (RFS) requests and provides additional storage space to MPSS and ADSP processors by storing data in the High Level Operating System (HLOS) fileystem. Signed-off-by: Pradeep P V K <pradeep.pragallapati@oss.qualcomm.com>
Add DEFAULT_PREFERENCE for bitbake to choose the prefered build recipe between tqftpserv and tftp-server. Add RPROVIDES to support virtual provider pattern. This allows distro package manager to choose between tqftpserv and tftp-server implementations using PREFERRED_RPROVIDER_virtual-tftp-server. Signed-off-by: Pradeep P V K <pradeep.pragallapati@oss.qualcomm.com>
Replace hardcoded tqftpserv package with virtual-tftp-server to allow distro configuration to choose between tqftpserv and tftp-server implementations. Signed-off-by: Pradeep P V K <pradeep.pragallapati@oss.qualcomm.com>
Set PREFERRED_RPROVIDER_virtual-tftp-server to tqftpserv to avoid build warnings when both tqftpserv and tftp-server are available in the same layer. This assignment allows distros and users to override the preference if needed. Signed-off-by: Pradeep P V K <pradeep.pragallapati@oss.qualcomm.com>
|
|
|
||
| # Set default runtime provider for virtual-tftp-server to avoid | ||
| # warnings when both tqftpserv and tftp-server are available | ||
| PREFERRED_RPROVIDER_virtual-tftp-server ?= "tqftpserv" |
There was a problem hiding this comment.
This should not be necessary with the DEFAULT_PROVIDER, unless I misunderstand something.
There was a problem hiding this comment.
It works, but bitbake will issue a note asking the user to consider setting up a PREFERRED_RPROVIDER anyway.
There was a problem hiding this comment.
Then do we need the DEFAULT_PROVIDER at all if we define PREFERRED_PROVIDER?
There was a problem hiding this comment.
That's something to check.
There was a problem hiding this comment.
DEFAULT_PREFERENCE allows to de-select the version provided by a specific provider. So, I think both are not same. In this case, DEFAULT_PREFERENCE might not be helpful.
|
@Pradeep-pvk what is the path mapping that is used by these binaries? |
|
Also, does tftp-server support compressed firmware? |
The current binary uses the below path mapping For MPSS For ADSP |
yes it supports compressed reads. |
Why are you providing two paths for each entry?
What kind of firmware gets requested? wlanmdsp or something else?
This should be /var at least
Ramdumps definitely should not go to /etc.
What is the usecase for those items?
What kind of firmwre is being loaded by ADSP?
Same comments apply. What about CDSP and SLPI? |
The first is the normalized path for each mapping path and the second is a symlink of the first path.
it would be mostly modem firmware blobs/mbn's, oem mbn's, etc.
yes, it was under /var initially but due to some new requirement( OSTree for performing OTA) we were asked not to install anything under /var, /opt, /mnt, /home paths.
For /hlos - AFAIR, it will be will be used by location services, xtwifi download use-cases, OIS PD etc. For /shared - contents or data that can be shared among multiple subsystems would be placed under this path.
it would be audio calibration data, sensors registry etc.
yes, it can have similar mappings for CDSP and SLPI as well but currently it is limited to ADSP and MPSS on QLI. |
|
@Pradeep-pvk please fix rhe formatting of your comment. All your answers for stuck in the quoted text. It's hard to visually separate them. |
yes, formatted my comment. please check and let me know, if still having the issue. |
Why do we need a symlink?
Okay. So there are two usecases:
Loading
Rootfs can be read-only. All state files must go to /var (and anyway, you are not installing anything, it gets stored there, like any other stored file).
Still, nope.
I haven't seen this. Which platforms use this? Are they to be provided by the host or are they stored by the modem?
What is it? Is it stored by the modem or is it to be provided by the host?
calibration data - unsure. Sensors registry should be loaded from /usr/share/qcom/SoC/Vendor/Device/sensors/.
LImited by whom? We support SLPI on MSM8998-SM8450. |
Each HLOS (LA, Windows) have their own way of storing executable-images, data, ramdumps and also their own file-access-permission modes. So the RFS-modules files location in the HLOS will vary from HLOS to HLOS and hence the symlink to the actual path on device for a given normalized rfs path.
The tftp_server can still support this—it is capable of handling absolute paths requested by the client, as shown in the example above (e.g., wlanmdsp.mbn), where clients prefix the request with /root//
AFAIK, /hlos RFS path will be used on all platforms. The above is just an example (RcvdTdpFile.txt) taken from a Pinnacles MBB target and unsure of this file presence/creation on other targets.
/shared - This folder will be shared by all the qcom-processors (MPSS, ADSP and other peripheral processors).
tftp_server can still support the above path, provided the client requests the actual absolute path prefixed with root.
sorry for the confusion, yes it supports SLPI and CDSP paths too. |
Well, no. this should be handld in the server code or via runtime configuration. symlinks are an unnecessary complication.
It seems you don't understand. There are existing usecases, loading wlanmdsp.mbn. Modem DSP is not passing an absolute path. It is passing /readonly/firmware/wlanmdsp.mbn (with several variants). This must be handled by supplying the correct file (which is platform-dependent). Not to mention, that you you didn't specify
Please. Come up with actual issues relevant for the QLI targets.
I don't see it being requested on any of the systems I've tested. Could you please provide additional details?
I think you are approaching it from the wrong side - the clients (sensors firmware) already exists. We must support existing systems rather than enforcing "supply this path" resrictions on them.
|
This has been the existing approach followed in tftp_server across all qualcomm chipsets spanning multiple BUs, and it is also a commercialized solution.
To me, this appears more like an optimization - Why is the file selection handled by the server instead of allowing the client to choose the required file?
Sure, i will try to provide information on QLI targets.
You may be testing on systems where tqftpserv is enabled. This behavior is typically observed on systems running the tftp_server. As mentioned earlier, this file is a handshake file created by the tftp client during TFTP client–server communication.
The current tftp_server works in this model and was a commercialized solution. The clients (sensors firmware) is responsible for supplying the correct path to enable the server to perform the requested read or write operations.
|
-ENOTANARG. There are enough ugly solutions in the "commercialized" software. Starting from Qualcomm downstream DT files.
You can read the source code of tqftp server and answer your question.
Again. I haven't seen the firmware requesting this file. Would you have any details on which platforms it is actually used by the DSPs?
Exactly. The sensors registry is platform-specific, so it should go to the device-specific location, which logically fits to go together with the DSP binaries. But.... It makes me ask a different question. On the current platforms as far as I know the registry is being loaded by the sdsprcpcd directly, without involving TQFTP. Which platforms are you talking about here?
Bingo. You didn't bother to look, how tqftpserv is integrated with the rest of the system.
|
lumag
left a comment
lumag
left a comment
There was a problem hiding this comment.
After discussion with other meta-qcom maintainers and with the platform maintainers.
- tftpserver doesn't seem to provide benefits for the QLI platforms, which are usually modemless and load sensors registry and sensors configuration through sdsprpcd,
- it will break the usecase of loading ath10k wlanmdsp.mbn firmware file from the existing known locations,
- it seems to provide DSPs with the access to undocumented locations, including possibly complete rootfs access, leading to possible security implications.
As such, I'd kindly ask to continue discussion with @Bamse and @quic-kdybcio regarding missing tqftpserv features and spend efforts on improving tqftpserv instead of trying to push and plumb the tftpserver into meta-qcom.
6 participants
Add new prebuilt recipe for tftp-server to fetch Qualcomm properitary userspace application from artifactory. This application serves the Remote Files System (RFS) requests and provides additional storage space to MPSS and ADSP processors by storing data in the High Level Operating System (HLOS) fileystem.