Async Tasks

[LeonanCarvalho]

Hello,

I found that brilliant project and I was very curious about that, I removed the dust from my old lineage2 DVD and installed it again, after solving a lot of problems to run such an application on windows 10, so nostalgic. I'm into this private server stuff since I had out of money to pay for NCsoft subscriptions and its means at least 12 years.

Besides Lineage2 passionate, I`m a software engineer, and I love to plan, discuss, design, and build.

My first impression and question were about NPC animation, that's not natural and appears to be in a single task not random.

One of the weirdest things in l2j is the npc walk animation, it happens because they don't have an NPC server to control NPC AI. have a separated NPC Server (as l2off do ) is good for scale, but not for convenience for sure, at least without object storage and a SharePoint.

It can be solved by implementing a random animation, and I did:

const MovePercent = .03; //only 3% of all NPCs should walk (this number may be smaller than it)

function shuffle(unshuffled) {
  return unshuffled
  .map((a) => ({sort: Math.random(), value: a}))
  .sort((a, b) => a.sort - b.sort)
  .map((a) => a.value)
}

let npcList = shuffle(world.getNpcList()); //Shuffling will grant us a random movement.
let maxNpcMove = Math.ceil(MovePercent*npcList.length); 

for(let i = 0; i < maxNpcMove; i++) {
//...

Doing a lot of random calls in the main thread may not be a good idea, in fact, all project is running into 2 threads, game server, and login server.

Node is a server-side platform based on asynchronous I/O, which means you don’t need threads (which makes things a lot easier). Concurrency can be a very hard problem to solve. Having many threads accessing the same memory can produce race conditions that are very hard to reproduce and fix.

Since the release of Node.js v10.5.0, there’s a new worker_threads module available, and it has been stable since Node.js v12 LTS, it means that actually, we can run things in parallel, but we don’t create threads, and we don’t sync them. The virtual machine and the operating system run the I/O in parallel for us, and when it’s time to send data back to our JavaScript code, the JavaScript part is the one that runs in a single thread.

This will make a lot of sense when a real database be implemented, (probably MongoDB if I`m not wrong), and yes Lineage2 even c1 will probably have a lot of random and async tasks.

The biggest problem, if one task crashes the process, it will leave all tasks sent to the same process unfinished.

In order to fix these problems, we need multiple forks, not only one. But we need to limit the number of forked processes because each one will have all the virtual machine code duplicated in memory, meaning a few MBs per process and a non-trivial boot time.

So, like database connections, we need a pool of processes ready to be used, run a task at a time in each one, and reuse the process once the task has finished.

Well, that may solve the problem but, will use a lot of memory, creating a Worker — even though it’s a lot cheaper than forking a process — can also use too many resources depending on your needs. In that case, the docs recommend you create a pool of Workers. You can probably look for a generic pool implementation or a specific one in npm instead of creating your own pool implementation.

But let’s see a simple example. First, we are going to implement the main file, where we are going to create a Worker thread and give it some data. The API is event-driven, but I’m going to wrap it into a Promise that resolves in the first message received from the Worker:

3b7ce87

Well, to make a game server to a server based on TCP connection you probably will need to work with the async core.

Have fun

[LeonanCarvalho]

Note:

Work with workers is not about threads itself, but to parallelize the processing and take advantage of multiprocessor servers
Also interesting is take a look into Cluster Module https://nodejs.org/api/cluster.html

[space2pacman]

Hello, I am very glad that you not only found my project, but also participate in it. It is doubly nice that you are also a developer of l2jbrasil.

Yes, on Windows 10 there are problems with the launch of the game, so in the near future I will prepare a ready-made distribution of the game.

As for the NPC single tasks, they are written for the test. As a temporary solution. In the future, there are also plans to have a separate server for NPCs like PTS (l2NPC). Separate server for caching (l2cached) etc.

Database will probably be MongoDB.

At the moment, after a long break, I bought myself a candy bar for comfortable testing. I will also draw up a plan for development. I will add versioning. I will add README.md. And I will indicate the points that I will need to finish in order to release the first version.

Thanks for the information about the cluster.

Thanks for joining the project :)

[VardanNersesyan]

Hi, how I can take a part in this project ?

[space2pacman]

Yes, sure :)