A/B split testing for Recipes
Case number: | 845813-987271 |
Topic: | Game: Tools |
Opened by: | ChristianK |
Status: | Closed |
Type: | Bug |
Opened on: | Sunday, January 24, 2010 - 21:53 |
Last modified: | Saturday, July 13, 2013 - 03:42 |
When creating a recipe there's a problem that you really don't know whether you should put 10, 15 or 20 into a function like set_behavior_clash_importance.
It would be nice to be able to write a recipe that randomly chooses one of the three numbers every times the recipe runs and afterwards uploads information about the score change that the recipe produced and the time that the script took to complete.
This would allow evidence based fine tuning of the parameters of recipes.
So, a list of ALL possible variables and their range would be nice.
Normally, "real" random numbers are possible in LUA (as far as random numbers are general possible on a computer anyway), but unfortunately, the LUA standard library is not implemented into the game at this time.
Themarquis and me use pseudo-random-numbers.
Anser here if you want to know more.
Of course it's possible to generate pseudo random numbers by writing a bit of code.
Here the problem is however not only about generating random numbers but also about actually measuring results of a script.
At the moment a script can't write a text file or upload data to the cloud and therefore it's nearly impossible to compare different values.
True.
We need file access by scripts.
The problem with full file access is that it might give us the possibility to blast the sequence in a real database.
When we get puzzles where the 3D structures is already known in uniprot a script that looks up the real sequence has an unfair advantage.
It's a huge problem in bioinformatics to make sure that you don't use information of the 3D structure of a protein that's already known to make a prediction about the 3D structure of the same protein.
Therefore the information exchange between the script and the rest of the world has to be limited a bit.
At the same time it's crucial to actually use the information that gets produced by running the script to improve the script for the use in future puzzles.
It would also nice to gather the information from all uses of the script whether the script runs on the computer of the author of the script or whether it runs on the computer of someone who downloaded it via the shared recipe site.
Status: Open » Closed |
Type: Suggestion » Bug |
not doable
Ah, I'm sorry for multiposting. It somehow didn't show at first.