[8.6] Create delete-serverless-dir-in-backports.yml (backport #5300)

[8.6] Create delete-serverless-dir-in-backports.yml (backport #5300) #9