Baby Littering and Global Goblin Growth

[squealingpiggies]

I noticed something interesting after watching the world play out for almost a century: eventually everyone was goblins. After some investigation I think is caused by a combinations of three things:

• warband recruitment from current location leads to trader characters picking up all sorts of different race warriors

• warriors being checked for reproduction in whatever province they are currently in

• and finally, the population growth function that just drops the baby in the current province
  https://github.com/Calandiel/SongsOfGPL/blob/dcde1b7008d88a7384993016851efebdd3fa0d02/sote/game/society/pop-growth.lua#L80

This is what I like to call Baby Littering, warriors will march around and provinces will check females for growth. If a baby is made it is just dropped at the current location as it's home even if it is different from the parent. Couple this with parents not keeping track of their children (only children keep track of their mother) and you effectively have tribes potentially picking up orphans from any traveler.

This wouldn't be much of a problem if all races reproduced similarly but Goblins have a rate of 4x humans (8x elves) so they end up over producing and eventually become the dominant race once they drop a couple of babies in a province and breed like rabbits forcing the province to uplift pops to notable characters, which more often then not is a goblin now; eventually squeezing out every other race from their own provinces. If the simulation was left running long enough, the whole world just becomes Goblins.

starting a new world and looking at race map mode

and then after about a single goblin's lifespan

The quick fix is probably just spawning the child in the parent's home province. Alternatively, if mothers tracked their children they could checked in the travel calculations to have the child move with their parent if the not in their home province, forcing the child to be dragged along with the mother until she returns home.
The quick fix also has an issue with using warbands to facilitate "migration" where you can go to other provinces to recruit warriors and bring them back to your home, remove funds from the warband and watch as the foreigners take up jobs in the province but never change their home province so any child born from these will teleport to wherever the parent was was born where the latter solution solves this but at either memory cost in a table or processing by looking for through province children when traveling from nonhome provinces.

[Calandiel]

How about making people in active warbands be unable to have children instead? It'd give the player a reason to not always be in one

[Calandiel]

In all honesty, there's also a problem of not having epidemics in the game. I think goblins having higher population pressure is fine but I agree that as is they're way too good at spreading

cc @ineveraskedforthis

[ineveraskedforthis]

Yes, this is an interesting problem, I think another way to solve it is to implement an option to get rid of unwanted goblin children in some way. In my opinion, if goblins are left unchecked and tolerated by their neighbors, they should eventually outcompete all other races. And i think that epidemics will make goblins (and still not implemented vermens) only stronger, because they are supposed to have high resistance to infections.

Essentially, I think this problem will solve itself after development of race-specific laws which will allow tribes to target goblins and other dangerous races and keep their population in check or enslave them.

Short-term solution like allowing pops to have children only when they are in their home province will work for now, but IMO this is quite an interesting emergent effect, which would be fun to have when other systems will be in place.

[Calandiel]

Impact of disease is hard to predict. It's true that better disease resistance would help prevent high death tolls but higher population density would also mean more frequent disease outbreaks.

I agree, though. We really need some kind of Stellaris-esque law system to allow controlling goblins by force.

[squealingpiggies]

I feel like the point about literal Baby Littering is being overlooked. The goblins themselves wouldn't be as bad if their children were born in or travel to the parent's home. The problem is that mothers just don't care (keep track, pull along, feed, etc) about their kids and just drop the baby and walk off like it never happened. I don't think its really the goblins fault as much as it is just that one line of code with no follow-up leading to the problem.
I don't think we should prevent warriors and characters from potentially reproducing anywhere (do we not want traveling characters to have trysts that lead to children?) but we should make sure that the children move with their mother.

If the logic isn't too heavy I think traveling should grab the characters children that can't work and are not in their home province. Additionally, -- community helps children as well code should either pull from the home province or check that they are in their home province to prevent foreign orphans from being cared for by the locals.

[ineveraskedforthis]

Nah, the problem is real, but the thing is that this Baby Littering is quite a fun consequence of that one line of code. But it's definitely way too common. Your suggestions are quite good and shouldn't be too hard to implement. We should track children anyway just for the sake of potential succession rules, so it's probably a good chance to implement it.

[squealingpiggies]

So after investigating the fix with parents tracking children, the follow needs to be done

• add a table to POP that holds children
• change pg.growth(province) to insert child into parents children table and set child's home to parents home
• potentially change st.run() to assign starting pop parents and children where applicable
• change satisfy_needs(pop, pop_table, free_time, savings) to only give children from tribe wealth when present in its home province
• change prov.Province:set_home(pop) to recursively call on non-adult children
• change prov.Province:transfer_character(character, target) and prov.Province:transfer_pop(character, target) to recursively call on non-adult children that are not in their home_province
• possibly change effects.death(character) to remove itself from parent and children tables (if we don't want to keep track of dead people)

Is this a workable solution? And should the cutt-off for the children be teen_age or adult_age? If the former the pop needs to be fired and then consider teens in warbands. Should they be also be pulled or left in the warband? Alternatively we could also give teens a "choice" by moving them only if they are non-drafted and unemployed.

Also while pg.growth(province) is being looked at, I noticed that teens can't get give birth, is this something that should be changed? Perhaps teens birth chance should be scaled similar to free_time for children in satisfy_needs. I kind of want to see the recursive set_home and/or transfer_pop pull a character, their children, and one (or more) grandkid(s) to a new province.

[Calandiel]

These sound reasonable. Wrt dead people, we have to cut the trees somewhere because otherwise the garbage collector will never free memory.

That being said, the precise behavior of functions like set_home when taking children into account should probably be made into a culture trait and checked for before running any actual logic.
It seems like a good opportunity to add different kinds of kinship, for example.

[squealingpiggies]

What kind of kinship do you mean? Are we talking marriage and spouses like an additional table, or more like family units?
Right now, I don't think characters give birth at all and the only way new nobles form is from uplifting pop. Some form of relationships could be helpful for calculating which characters should be getting pregnant, which is something pop should also be tracking so as not to delivering monthly.
If something like marriage and/or families are implemented, I think they should be reserved for characters and normal pop should only use the table to track their direct children while they are dependent and then forget about each other when becoming adults/teens.

[Calandiel]

I mean kinship as in anthropology: https://en.wikipedia.org/wiki/Kinship
In particular, "the big six", but there's plenty more that were more niche and would add a lot of juice to a de facto fantasy game.

We are talking family units but the implementation will have to depend on the importance of the person at hand - we will need them for dynastic succession (see CK3 and Imperator), but storing a bunch of extra tables on every pop would be extremely wasteful, as you mentioned.

[squealingpiggies]

So we are talking about tracking children, parents, and, spouse(s) at the least? Do we want to hold onto dead parents until all children are dead too to keep track of siblings or should there be another data structure to hold family units (potentially combining parents, children and spouses into a single class)?
And what kind of cultural traits are we looking to implement with this initially with this? Should we just look at decent for succession or should family unit type also be considered?

[squealingpiggies]

I implemented a bare-bones child-parent relationship using a children table that assigns children to parents on tribe generation and pop birth and culls the relationship on death. I also set the pop growth function to share their parent's home province and have the transfer_pop function check the pop for any children and have call the transfer function if they are contained in the all_pop table and they are not in their home province, part of a warband, and employed. This will pull any unemployed decedents with transfer_pop and should prevent baby littering by helping to move foreigners along if they are not helpful to the tribe.
Edit: added comparison runs
Initial world:

After a similar amount of time as the first post, unaltered:

After similar time with pop_transfer pulling children:

This growth would probably further reduce with preventing tribes from donating to foreign children and checking if there are at least 1 male and 1 female of a race to reproduce.
After similar time with additional requirement of a male and female of same race in province:

Edit: one last run, with an additional hackish prevention of recruiting different races into a warband

This last screen shot seems to show that the real problem is that everyone decides to hire goblin warriors, since there are always so many unemployed, which then come back home and start popping out babies. Perhaps a cultural trait linked to racial, cultural and religious aversion could do a better job at reducing the goblin menace than child tracking. Alternatively, perhaps reducing the goblin numbers at start could curb their growth significantly since there are so few people in tribes at start; maybe tribes just need the time to get a stable demographic to compete against a couple goblin mercenaries.

[Calandiel]

Could you make a PR? It'd be easier to me to analyze is that way.

Another approach could be making goblins much weaker at combat and not letting them to just settle down in settlements that don't explicitly allow "non accepted" pops. Similar to how Vic2 handles migration for some government types.

[squealingpiggies]

It looks like the real issue is that all races, cultures, and faiths intermingle without restriction that allows the traders and their warband to visit any province and start recruiting from it and/or letting their current multiracial army drop babies. With goblins only taking a couple of years to raise and get a job, the parent pulling stops working on more on them since labor becomes a bottleneck for everyone except goblins after just the first decade or so.
The idea of nonaccepted pops could be handled in the culture by adding opinions on being of different race, culture, and faith. This could keep some of the moving around, at least within race and culture group. It could easily be used to restrict employment and warband recruitment which would restrict it even more. It could also be used with the AI raid and trader targeting logic on who to favor for each.
These cultural opinions on differences could simply be a boolean? to represent a hard limit (true), opinion modifier or reduced chance (false), no opinion (nil) and be randomized on spawn generation so some places could experience goblin overgrowth while others wouldn't.
After my test runs, I am starting to believe that the real issue it that tribe too easily intermingle. There should be some natural barrier to overcome so that these interaction only happens if certain conditions are met and I think cultural restrictions on interactions could work nicely for that. It just might be even more effective at managing goblins than pulling children.
Edit: Goblin growth tamed with a single line of code! Adding a hard limit to the travel decision AI targeting to require the province or realm leader be of same race, culture and faith in:
https://github.com/Calandiel/SongsOfGPL/blob/ab6201ced1af3b94876d468f0d06a25602ca231b/sote/game/raws/decisions/travel.lua#L109

Initial race distribution:

After 42 years:

Comparing the minimaps between the two can see instance of most races dominating with some other race province in areas but this is only due to the tributary-overlord targeting which doesn't check race, culture and religion.

[squealingpiggies]

In regards to #179, I also have an alternative option I have been playing with that doesn't even pull children yet and does a much better job at controlling warriors dropping babies here by simply randomly making some tribe intolerant and refuse to hire different race/culture/faith warriors or travel to their provinces (but not stop others from traveling to their provinces). These are tied to the culture and randomly assigned in the on world generation between hard refusal, general dislike, and ambivalence towards a other races, cultures and/or faiths; in any combination.
Some screenshots so far of a world I have been running some simulations on:

CONTROL RUN: initial culture definition and randomization but otherwise unaltered. (40% isolationist, 40% exclusionist, 20% ambivalent)

FIRST CHANGE: Isolationists don't target neighboring realms where their race/culture/faith is dominate or is the ruler (later added a check for realm primary as well but not for this run) and will weight all targets based on their cultural opinion of differences.

SECOND CHANGE: Warband leaders and recruits follow their culture and refuse to hire pops if they are not eligible.

The biggest change was just having the traders not run around randomly and stick to the familiar. The second addition seemed to have little effect. I haven't even touched baby littering and it seems much more controllable.
It still needs the cultural attitudes to be used for raiding and tribute to see how xenophobia shapes the world (and hopefully keeps at least a few pockets of each race goblin free).

[Calandiel]

That seems intriguing. I'll check it out after work today!

[squealingpiggies]

I realized that my previous hard limit was probably not the best approach, so I attempted to implememnt it using a cultural trait system based off how character traits work. I have 5 traits: isolationist (malus to different race/culture/faith), racist (bonus or malus based on race), bigoted (bonus or malus based on difference from culture), fanatical (bonus or malus based on faith), and avarice (increased perceived value of money). These were then implemented in as flags in generic_event_opinion which was then used to determine how willing traders were to travel, warbands were to recruit, employers were to employ and pops were willing to be employed and switch jobs. The weights can definately be adjusted but it seems keep most races contained to there starting areas. The branch can be found here.
It still need to have the culture flag applied to all relevant calls of generic_event_opinion and to use the culture flag to weight raiding and tributary targets but it seems to do a good job at controlling goblin growth without hard limiting anything. And the weighting algorithms definitely could use a look over as they are currently magic numbers and stand-ins at best.
Screenshots from test runs:
Initial world:

Run 0 - control:

Run 1 - travel restrictions:

Run 2 - removed isolationist:

Run 3 - recruitment restrictions:

Run 4 - employment restrictions:

Run 5 - half bonus/malus with extended time:

Run 5 - final culture view:

[Calandiel]

I merged in your MR. We should probably keep this issue open for now

[squealingpiggies]

Yeah, while children tracking is helpful, I think it is only one part of the solution to goblins overtaking neighboring realms just by visiting and am still investigating a solution. I'm hesitant with both of the cultural restrictions with travel both because it restricts movement so heavily and touches on #145 (thus feels a little outside the scope) but I do like the added affect that the trait system has with restricting recruitment and hiring to make conditions less favorable for foreigners and fits nicely into the existing systems. Still, I am looking for a better solution that allows goblin traders to visit and maybe have some warriors put down roots but prevent goblins from gnobbling up all their neighbors. So with the merge, I did a rebase on my trait branch and ran a hundred years to see how the two worked together and did some digging.

Initial world:

After 100 years:

An elf realm:

With racist elfs:

And bigoted (cultural) goblins:

All the elf pop:

Sample of goblin pop:

Province nobles:

How it all started:

Running the world on master:

The most interesting thing I noticed from this is that all the elves were employed while most of the goblins weren't. Second was the different in satisfaction and life needs. the elfs were between 75-100+ needs while the goblins were stuck around 20-50. This was probably explained by the racist elves not wanting to hire goblins or the bigoted goblins not wanting to work for coish (elf culture). Since the goblins would all have their home set somewhere else, the children won't take from province wealth (which is probably why most are at ~20 needs) so why are they still overtaking this province?
My impression from this is that a goblin trader came in to the province and sat around long enough for the pop to grow enough to select a new noble and that happened to pick a goblin; which in turn allowed a goblin pop to be hired and generate more wealth that would only fuel their growth. Even if the goblins are starving, they still seem to be able to grow to adult age and reproduce.

Two simple counters could be done for this:

1. Make noble promotion use cultural weights to randomly select from tied best fit and/or limit noble uplifting to pop of primary race, culture, and faith. Potentially limiting the trigger to only look at the capitol's home_pop and weigh noncharacter to character instead of just comparing to number of people and character present.
2. Make goblins develop slower, as in take longer to reach adulthood and breed. This could potential lead to more starvation deaths before they start popping out babies and drastically reduce their growth in unfavorable conditions.

Regardless of these to idea, it also occurs to me that so simple neighbor targeting pop migration based on at least needs should be added to handle these pops that are living in harsh conditions. The calculations would also be useful for determining when an ai tribe would migrated, as I have now seen a few tribes generated alone in desert mountains and slowly dwindle instead of moving to nicer neighboring provinces.

If you have any other thoughts or ideas on possible avenues of investigation, let me know.