[REQUEST] Suggestion to improve perf / HLSL

[yoyofr]

Please confirm the following points:

• This question is NOT about the Android apps in the Play Store
• I have searched the project page to check if the question was already asked elsewhere

Topic

General Request

Your Request

Hi,
I'm currently integrating ProjectM in an iOS app, Modizer. it's working quite well but I'm facing regular stuttering when switching preset. I went through #545 and did some profiling.
Issue seems to be the following call in HLSL which eats a lot of CPU in the preset loading process:
static inline double iss_strtod(const char * in, char ** end) (line 67 in hlslparser/src/Engine.cpp)
It seems the overhead is huge.
I tried to just call regular C strtod function instead and it seems to work fine so far, stuttering has gone. Not sure it is important to manage the C++ to C locale for that part.
Edit:spoke too soon, stuttering has reduced but not fully, will do more troubleshooting.

[kblaschke]

The locale is actually important, as these functions - by default - use the currently set locale to parse decimal punctuation. E.g. with my German locale, "1.234" would be parsed to "1234", as . is the thousands separator and , the decimal one. It is not easy to change the locale for the C library functions as it is with the istringstream used by this implementation, so I guess this is why Unknown Worlds chose to implement it this way.

One remark on iOS/tvOS: Please be aware that you cannot publish the app using projectM in the Apple Store, as Apple's Store TOS aren't compatible with (L)GPL. You can only use projectM in sideloaded apps.

[mbaetgen-wup]

I am seeing the stuttering as well. I can confirm that it happens on the frame when the next preset is being loaded. It is not significant with all presets, but seems reproducible with certain presets.

[mbaetgen-wup]

I'll take a closer look.

[yoyofr]

The locale is actually important, as these functions - by default - use the currently set locale to parse decimal punctuation. E.g. with my German locale, "1.234" would be parsed to "1234", as . is the thousands separator and , the decimal one. It is not easy to change the locale for the C library functions as it is with the istringstream used by this implementation, so I guess this is why Unknown Worlds chose to implement it this way.

One remark on iOS/tvOS: Please be aware that you cannot publish the app using projectM in the Apple Store, as Apple's Store TOS aren't compatible with (L)GPL. You can only use projectM in sideloaded apps.

thanks for the reply.
My assumtion / locale is that as the function is used only for the HLSL parsing, i.e. shader sourcecode, it doesn't require to manage localized decimal numbers.

Regarding iOS/tvOS I'm aware but I also see that it is very common to have (L)GPL based app available that follow the spirit, i.e. sourcecode available with capacity to compile. VLC, ScummVM, Retroarch, ... are all available for iOS to name a few.

[kblaschke]

thanks for the reply. My assumtion / locale is that as the function is used only for the HLSL parsing, i.e. shader sourcecode, it doesn't require to manage localized decimal numbers.

It parses the whole shader source including numbers, so the parser locale is relevant. Otherwise, floating-point numbers would end up with the wrong value if the decimal point is interpreted as a thousand separator. In the GLSL generator on the other end, this is the same thing. So if you deal with parsing numbers from text, the locale is quite important.

Regarding iOS/tvOS I'm aware but I also see that it is very common to have (L)GPL based app available that follow the spirit, i.e. sourcecode available with capacity to compile. VLC, ScummVM, Retroarch, ... are all available for iOS to name a few.

Just a heads-up, because the original projectM app made by Peter Sperl, a former projectM dev, was actually removed from the Store by Apple because of this issue. You can try to get it through their review, but be aware that it may not pass, or Apple might remove the app at a later point without notice.

[yoyofr]

Ok but the shader source code wouldn't have localized decimal numbers, no ?
I mean in presets files, I assume a float would be always C style, i.e. 1234.56 and not 1234,56.
What would be the use case where you have to deal with different locals when parsing hlsl shader source code data ?

[kblaschke]

Ok but the shader source code wouldn't have localized decimal numbers, no ?

Exactly.

I mean in presets files, I assume a float would be always C style, i.e. 1234.56 and not 1234,56. What would be the use case where you have to deal with different locals when parsing hlsl shader source code data ?

The "use case" is that, by default, the number parser function in the C library will use the current locale configured by the app user to parse the number format. Which, if set to "de" for example, would parse 1234.56 in the shader code to a float value of 1234560.0, which is clearly the wrong value by a factor of 1000.

This is why you have to explicitly set the locale to C to use english/US decimal notation for parsing, which is the correct one, and independent of the user's system settings.

So if you say we should remove this locale setting code, and just use the defaults, we'd have to tell every single projectM user to set his OS number locale to C or en_US, which simply isn't practical, right?

[yoyofr]

Sorry I realize I forgot a very structuring part: standard C strtod is generating a localized number. I was assuming it wasn't 😟
So I guess the best would be to replace it with a simple and optimized strtod like function that generate C style double.
I will work on it

[yoyofr]

I've done some tests using fast_float lib, it's a locale independant optimized version of strtod and seems to be a standard.

Doing quick benchmark of ProjecM::LoadPresetFile function (using clock() and measuring the execution of 'StartPresetTransition' call, it divided by 6 execution time :).

I tested loading the same 10 presets files from the Cream of the Crop pack.

On a macbook air M2, I had an average of ~120k to ~130k of ticks with the current code.

With fast_float::from_chars, it decreased to an average of 20k ticks.

fast_float can be integrated easily as just a header file and the modification to Engine.cpp are limited to String_ToDouble like this:

double String_ToDouble(const char * str, const char * strend, char ** endptr) {
// return iss_strtod(str, endptr);
double result;
auto answer = fast_float::from_chars(str, strend, result);
if (answer.ec != std::errc()) {
if (endptr) endptr=(char)(str);
return 0;
}
if (endptr) * endptr=(char * )(answer.ptr);
return result;
}