Non-blocking code?

[pcace]

Hi there, i am trying to figure out if my code is non-blocking. i have a fastify server which serves a service which downloads images in a certain area, transforms and reprojects downloaded images and then sends them back. the issue i am having is, that it seems to be working very sequentiell.
i am now wondering if it has to do with the way i have implemented gdal-async. i have read

node-gdal-async/ASYNCIO.md

Line 4 in 7e02bd4

but still cannot tell if i have followed it correctly.

maybe someone could take a look at this, and tell me if there is a blocking part in it?

{
  //... handle query and bbox calculation ... //
  const imageBufferWMSDownload = await fetchWithRetry(url.toString());
  // .. handle chaching etc ... //

  // Write the image buffer to in-memory file using gdal.vsimem.set (zero-copy)
  const vsimemImagePath = `/vsimem/${taskUUID}_${cacheKey}.png`;
  gdal.vsimem.set(Buffer.from(imageBufferWMSDownload), vsimemImagePath);

  const geoRefOptions = [
    "-of",
    "GTiff",
    "-a_srs",
    wmsCrs,
    "-a_ullr",
    // ulx uly lrx lry
    nwImageBoundsWmsCrsOffset[0].toString(),
    nwImageBoundsWmsCrsOffset[1].toString(),
    seImageBoundsWmsCrsOffset[0].toString(),
    seImageBoundsWmsCrsOffset[1].toString(),
  ];

  // Open the in-memory image file
  const ds = await gdal.openAsync(vsimemImagePath);

  // Save the georeferenced image to another in-memory /vsimem/ file using gdal.vsimem.copy
  const vsimemGeorefImagePath = `/vsimem/${taskUUID}_${cacheKey}.tiff`;
  await gdal.translateAsync(vsimemGeorefImagePath, ds, geoRefOptions);

  // tranform the whole image to Pseudomercator to be able to crop straight
  const ds_georeferenced = await gdal.openAsync(vsimemGeorefImagePath);
  const vsimemPseudoMercatorPath = `/vsimem/${taskUUID}_${cacheKey}_cropped_trimmed.tiff`;

  await gdal.warpAsync(
    vsimemPseudoMercatorPath,
    null,
    [ds_georeferenced],
    ["-t_srs", "EPSG:3857"]
  );

  //... some geometry voodoo ... //

  const vsimemGeoJsonPath = `/vsimem/${taskUUID}_cropPolygon.geojson`;
  gdal.vsimem.set(Buffer.from(JSON.stringify(cropPolygon)), vsimemGeoJsonPath);
  const vsimemCroppedImagePath = `/vsimem/${taskUUID}_${cacheKey}_cropped.tiff`;

  // Open the in-memory georeferenced image file
  const ds_cropped = await gdal.openAsync(vsimemPseudoMercatorPath);
  const cropOptions = [
    "-cutline",
    vsimemGeoJsonPath,
    "-crop_to_cutline",
    "-dstalpha",
  ]; // used for gdal.warp

  await gdal.warpAsync(
    vsimemCroppedImagePath,
    null,
    [ds_cropped],
    [...cropOptions, "-of", "PNG"]
  );

  const returnImageBuffer = gdal.vsimem.release(vsimemCroppedImagePath);
  const image = sharp(returnImageBuffer);
  const trimmedImageBuffer = await image.resize(tileSize, tileSize).toBuffer();

  // clean up the in-memory files by releasing them
  gdal.vsimem.release(vsimemImagePath);
  gdal.vsimem.release(vsimemGeorefImagePath); // const returnImageBuffer = readFileSync(splitImagePath)
  gdal.vsimem.release(vsimemGeoJsonPath);

  // some caching things.. and send it back...

  reply.header("Content-Type", "image/png");
  reply.send(trimmedImageBuffer);
};

is there anything what would prevent fastify to use this function in parallel?

any help would be great!! Thanks a lot!!

[mmomtchev]

I do not see any reason. vsimem methods are instantaneous, everything else is asynchronous. Either your image processing is very slow and blocking, either there might be a bug somewhere or something not working as it is supposed to.

There are various tools, including some dedicated to Express.js, that allow to measure the event loop latency:

https://blog.logrocket.com/top-tools-node-js-monitoring/

Try to identify the call that induces latency.

[mmomtchev]

And on the other hand, I am also curious if there isn't anything bothering you, since the whole GIS community is taking part in a gigantic extortion with the French police, judiciary and state institution revolving around the problem of one person?