Currently the following bugs are open:
Some of the reports seem to be closely related, some... maybe.
So I decided to start with something I really don't believe to be easily possible for roguelike games: I started to write automated tests using JUnit 4 and Hamcrest. Sadly after about an hour of test writing (never good as an afterthought, I know) I hadn't found anything. All the problem sources I had suspected weren't problems. All automated tests are running green.
While I already believed the bug to be fixed with getting rid of the "two weapon problem" mentioned in issue 992 other people reported a similar issue when just fighting with one weapon and a shield - so there must be something else I yet have to understand.
So I guess I'm missing something and that "something" IMHO is the problem with test automation in roguelike games: Roguelike games are extremely context intensive. A typical boring situation can be influenced by dozens if not hundreds of circumstantial factors (especially in JADE). I yet have to "get" the right idea on how to write reasonable tests for a roguelike game without reducing my productivity to zero - even in the face of that annoying regression bug up there.
In case my good friend Nils Wloka is reading this - suggestions are welcome (now that we are designing a test DSL for OpenSAGA and making great progress in that respect - I should have believed two years earlier ;-) ).
For now I will focus on issue 988 as shop-related issues usually are easier to reproduce.
I think 988 is produced when you drop two unpaid items of the same status. I've described in more detail in the 988 thread.
ReplyDeleteHonestly, I'm still far too ignorant on the game dev subjects needed to give good advice for this. However, that has never stopped me before!
ReplyDeleteWhat about somehow fusing together this testing method with an "ADOM II bot" whereby said bot serves as the Context Exploration Vector by which the testing bits can then examine as "things" are done by said bot?
Basically a very good idea but I' afraid that developing such a bot also takes months...
DeleteHmm, what about if said bot was designed to only play literally the first couple(2-3 if that) minutes of the game then suicide/replay? The thinking is that these sorts of things are cropping up in the very early, bare-bones game, since there isn't yet all the other mid-late game stuff nor other systems outright in place just yet. So, some little bot that tries a random, but logged, character build, then engages in just item equipping/manipulation/various shopping and such, then rerolls that again and again?
DeleteThat this bot would have much faster decision making than a human player should result in a "first couple minutes" even briefer than said human. It would be different than the various Borgs that Rogue and Angband have that are generally meant to "play"/win the game ideally due to the intentionally short lifespan and non-combat focus.
Something to think about if all else fails I suppose.
Are random number generators used in PC/shop inventory management? I think Issue 1009 might be related to 988. Even save-scumming, and repeating my steps exactly, did not guarantee identical error behaviour each time.
ReplyDeleteYes, they absolutely are. That's also the problem with testing this thing :-^ For me the RNG so far seems to play nice...
DeleteThis comment has been removed by the author.
DeleteYou already know what I'm going to say :P
ReplyDeleteI think if you fix the "troll brutes et al. attacking from afar" bug ( http://www.adom.de/forums/project.php?issueid=991 ), together with the change in level scaling, that's more than enough to justify a release. I'm currently not playing a lot due to the "attack from afar" bug that tends to kill all my chars.
The "failed to remove (...) from item list" bug may be annoying but it's not game-breaking. A 0.2.8 with that bug, but without the "attacking from afar" bug, would be *far* more playable than the current release, so I don't see a reason not to release it.
I'll try to hunt the bug though!
Both are very high on my list. The "failed to remove bug..." is more annoying because I believed it to be fixed a dozen times by now ;-)
DeleteI totally admit that the far range attack bug is more disconcerting...
Just fixed this one :-)
DeleteIs the monster code to a state yet where monsters can fight each other? If so, you might want to create an "arena": no player, lots of monsters fighting each other. Even if monsters can't fight each other yet, you could create a level and repeatedly place monsters and kill them, since at least one of the bugs happens when a monster drops its items on death.
ReplyDeleteTesting can be tricky, but here's a few ideas that might help.
ReplyDeleteFirst of all; rewrite the input function so that, instead of calling getchar (or the java equivalent), it calls a custom test function that merely returns a predetermined list of keystrokes. (In the current circumstances, a loop of 'reload-game-perform-actions-a-b-and-c' might help - assuming that the code that deletes games on load is temporarily deactivated for testing purposes) In this manner, you have a robot player built-in to ADOM without needing to take a month of to write it - have the function read the list of keystrokes to return from a file, and you've even got a customisable robot player. Not a very intelligent one, but you don't need intelligence here - you need simplicity.
If you want to stress-test the program, have the redone input function return random characters (either from a list of permitted random characters stored in a file, or any random character to test for the results of bad input). Have it write the random characters that it uses to a file, so that if it crashes you know what keystrokes caused the crash. Run it overnight, or while doing other stuff, and see what it's found when you get back to the computer.
I hope that's helpful...
When you remove an item from a list (in STL/C++) the iterators become invalid so you can't for instance use a simple for-loop to remove all items. Of course this is something you already know, but the bug might be something like that. Also I'm not sure if list works the same way in Java as in STL. Container routines can be deceiving, because they appear to be simple, yet there are surprisingly many ways to fail.
ReplyDeleteOr maybe an off-by-one error? I posted some test results in Issue 1047 which points in that direction.
DeleteI have a suggestion which I had myself used in my program (it was not a game). You define a class say 'Context'. You keep on maintaining objects which you feel are important. At every important step (like enter into a shop, wielding something) keep dumping this object to a file. Dump the same object when you get the same exception in a different filename. You can then do a diff between the two steps. You can encrypt these files if you feel that they will reveal the secrets. This is not fool proof but will mitigate the problem.
ReplyDeleteI think I now have fixed the bug. It seems to have been a combination of at least three problems. Hopefully this topic will never again rear its ugly head.
ReplyDelete