Usability improvements re: dummy coding contracts

[macdjord]

Preflight Checklist

• I have read the "Reporting Bugs / Requesting Features" section in the Contributing Guidelines.
• I understand that the issue tracker is for bugs in the game itself and feature requests, not for issues with my own scripts.

Description

There are a number of (relatively) simple changes with respect to dummy coding contracts (as created by createDummyContract()):

1. Allow the player to specify the name of the generated contract, or at least a specific non-root directory to generate it in. This will allow players to keep the dummy contracts out of the way, and not get them mixed up with any real contracts that might get generated on home.
2. The name of the generated contacts (or at least the default name, if you allow the player to specify it themselves as per the above) should clearly indicate that it is a dummy contracts, as well as what the contract type is. This will, again, help keep real and fake contracts separate, as well as making it easier to keep track of which contract is which if a player is working on multiple parts of their solver at once.
3. Dummy contracts should have an infinite number of attempts. They give no reward, so there's no reason to try to prevent players from using guess-and-check methodology, and if a player is trying to debug a solver, being forced to generate a new dummy contract with a different solution to test one halfway through is not helpful.
4. When submitting an answer to a dummy contract, there should be an option to not delete the contract afterwards even if it is correct.
5. Dummy contracts should have an option to show the answer. Again, there's no benefit to cheating, and when debugging, having a known good input/output pair to compare your work against can be invaluable.

• Note that using this button should not dismiss the contract; one of the use-cases here is, in combination with the above option to submit correct answers without deleting the contact, to test what different variations on a solution are or aren't accepted (e.g. different answer formats, giving answers in different orders, etc.).

[catloversg]

[1]: This is unnecessary. Real ccts cannot be generated on home.
[2]: It's fine to change the name to indicate the dummy contract, but it's unnecessary to change it even more to indicate the contract type.

With [1] and [2], it looks like you are trying to organize contracts while simultaneously developing solvers for different contract types. Differentiating contracts by name is usually only useful when using the UI (e.g., try to solve a contract manually via the terminal). You can build a script specialized for organizing contracts without relying on contract name. The contract APIs do not let you know which contract is the dummy, so I'm fine with changing the name for that. However, I don't think we need to change the name to indicate the type.

[3]: I'm neutral on this. Note that this should be implemented as an optional option in createDummyContract.
[4]: I'm neutral on this.
[5]: I veto this. Currently, if the player fails the contract 10 times and that contract allows getting the answer with getAnswer, we will print that answer. This is already a very questionable "feature" implemented in #2440. I did not veto it back then because it's still barely acceptable (the contract is destroyed at the same time we show the answer). However, I think this is the absolute limit of what we should do when it comes to showing the answer. To be honest, I don't think we should ever show the answer.

[macdjord]

[1]: This is unnecessary. Real ccts cannot be generated on home.

Oh? The documentation says that they can be generated 'on any server except ones purchased by the player, such as cloud servers and hacknet nodes'. This seems to imply that home is a valid location, since its not a cloud or hacknet server, and while it is owned by the player, it is not purchased by them. If it's not, the documentation should probably be updated to make it clearer.

In any case, being able to specify the name and/or location of dummy contracts would still be useful just for keeping things organized.

[2]: It's fine to change the name to indicate the dummy contract, but it's unnecessary to change it even more to indicate the contract type.

With [1] and [2], it looks like you are trying to organize contracts while simultaneously developing solvers for different contract types. Differentiating contracts by name is usually only useful when using the UI (e.g., try to solve a contract manually via the terminal). You can build a script specialized for organizing contracts without relying on contract name. The contract APIs do not let you know which contract is the dummy, so I'm fine with changing the name for that. However, I don't think we need to change the name to indicate the type.

My workflow for creating solvers for different contract types generally looks something like this:

1. Generate a dummy contract of the type
2. Open it from the terminal to read the instructions
3. Write code to solve it
4. Use a temporary script to load the dummy contract, specifying the contract name specifically, and pass it to the new code as a test
5. Repeat steps 3 & 4 until it gets the correct answer, possibly referring back to the contract instructions as needed
6. If it solves the contract correctly, use another temp script to generate another 100 contracts of the same type and see it it solves all of them, too
7. If it passes everything, edit my actual looping search-for-and-solve-contracts script to use the new solver when it finds contracts of that type.
   The problem is, if I have more than 1 dummy contract on home, in steps 3-5 there's no easy way to tell which one is which. More than once I've hard a solver generate absurd results (or even cause an infinite loop) because I accidentally put the wrong contract file name in and passed the data from one contract into a solver designed for a different type.

I agree that any script intended to systematically deal with contracts needs to look at the type as returned by openContract(), and indeed my looping search-for-and-solve-contracts script does just that. You'll note I didn't suggest changing the naming scheme for real contract to indicate the type. But by their very nature dummy contracts often have the interacted with manually or semi-manually, so being able to tell their type at a glance would be very useful.

Of course, if I could specify the filename myself, I could just make sure to give each contract I generate a suitable file-path.

[5]: I veto this. Currently, if the player fails the contract 10 times and that contract allows getting the answer with getAnswer, we will print that answer. This is already a very questionable "feature" implemented in #2440. I did not veto it back then because it's still barely acceptable (the contract is destroyed at the same time we show the answer). However, I think this is the absolute limit of what we should do when it comes to showing the answer. To be honest, I don't think we should ever show the answer.

Why? You have not explained how allowing players to view the solution would be a negative.

[gmcew]

3 and 4 I have stronger opinions on than CatLover for similar reasons to those given in #3008, which shares some similarities to this issue.

For 3: If your solver generates 10 unique wrong answers for a problem there's some major rework required anyway, the mild inconvenience of having to generate another dummy cct is just the symptom of a bigger issue.
Similarly for #4 the data was available, and was used to generate a valid answer. The player has both the raw data and the correct answer - the cct itself contains no additional value.

[macdjord]

For 3: If your solver generates 10 unique wrong answers for a problem there's some major rework required anyway, the mild inconvenience of having to generate another dummy cct is just the symptom of a bigger issue.

Debugging is an iterative process; having more than 10 bugs in your code before you get everything right is not unlikely. And it's not the minor inconvenience of having to generate a replacement contract that is the problem, but that the fact that you can no longer directly compare your results from previous versions with later ones.

Similarly for #4 the data was available, and was used to generate a valid answer. The player has both the raw data and the correct answer - the cct itself contains no additional value.

Entirely correct - for those contract type which have a single, unique valid answer. Some have multiple answers, or multiple ways to phrase the same answer, and testing to see what the contract does and does not care about is a valid use-case.

[catloversg]

Home is purchased by the player (ns.getServer("home").purchasedByPlayer). Technically, you cannot perform the purchase action because it's always granted to you by default, but that's just splitting hairs.

I consider viewing the answer a "cheat" (not in the literal sense). As you said, "when debugging, having a known good input/output pair to compare your work against can be invaluable", it's indeed invaluable, but it's not the game's job to provide this kind of invaluable information. That's too much "hinting" in my book.

[macdjord]

Home is purchased by the player (ns.getServer("home").purchasedByPlayer). Technically, you cannot perform the purchase action because it's always granted to you by default, but that's just splitting hairs.

Given I, an actual player, working from the documentation as currently written, understood home to be a valid location for contracts to spawn, it is not 'splitting hairs'; it is a failure to communicate clearly.

I consider viewing the answer a "cheat" (not in the literal sense). As you said, "when debugging, having a known good input/output pair to compare your work against can be invaluable", it's indeed invaluable, but it's not the game's job to provide this kind of invaluable information. That's too much "hinting" in my book.

The fundamental challenge of coding contracts - at least if you're trying to create an automatic solver script for them - is about understanding the problem and finding a (reasonably) efficient algorithm to solve it. No number of known input/output pairs will allow you to do that. What a known input/output pair will do is help you create an correct implementation of whatever algorithm you come up with, by assisting in identifying what has gone wrong. (Figuring out why it is wrong and how to fix it is still up to you, though!)

Designing algorithms to solve problems new and interesting problems is fun. Adapting code to fix a bug or to account for a complication you hadn't considered when you first wrote it can be fun. Banging your head against a wall because you can't even tell what or where the problem is in the first place is not fun.

[d0sboots]

I agree that "purchased by player" is a confusing area - in general, but this is a particular symptom of it. In this case, changing the docs in the relevant place to indicate that we're including home in "purchased by player" ought to fix it.

Entirely correct - for those contract type which have a single, unique valid answer. Some have multiple answers, or multiple ways to phrase the same answer, and testing to see what the contract does and does not care about is a valid use-case.

All contracts have a single answer. The most wiggle-room they have is in formatting, and that is typically for user convenience - i.e. extra whitespace is allowed. The problem description is pretty clear on formatting - if it's not, that's a bug, but I think we've squashed those particular ambiguities by now.

Designing algorithms to solve problems new and interesting problems is fun. Adapting code to fix a bug or to account for a complication you hadn't considered when you first wrote it can be fun. Banging your head against a wall because you can't even tell what or where the problem is in the first place is not fun.

This is why CCTs are an optional part of the game. By the nature of how they work, it's entirely possible to get stuck on one. They are much more "single-track" and less open-ended then the rest of the game's challenges.

[gmcew]

On the updating documents issue, the wording was updated about 3 months ago but hasn't yet made it to stable:

They can appear on non-darknet servers that are not owned by the player (Server.purchasedByPlayer is false).

[hexagonrecursion]

All contracts have a single answer

Incorrect. Examples:

1. GenerateIPAddresses accepts an array of strings as an answer, but ignores the order of elements in the array
2. In Proper2ColoringOfAGraph you can replace 1 with 0 and 0 with 1 (i.e. swap the two colors)
3. In LargestRectangleInAMatrix there could be multiple largest rectangles:

   [
   [0,1,0],
   [1,1,1],
   [0,1,0]
   ]
   

[d0sboots]

I rightly stand corrected.